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ABSTRACT 


We  show  how  to  formulate  many  continuous  timc-and-space  search  problems  as  gen¬ 
eralized  optimal  control  problems,  where  multiple  searchers  look  for  multiple  targets. 
Specifically,  we  formulate  problems  in  which  we  minimize  the  probability  that  all  of 
the  searchers  fail  to  detect  any  of  the  targets  during  the  planning  horizon,  and  prob¬ 
lems  in  which  we  maximize  the  expected  number  of  targets  detected.  We  construct 
discretization  schemes  to  solve  these  continuous  time-and-space  problems,  and  prove 
that  they  are  consistent  approximations.  Consistency  ensures  that  global  minimiz- 
ers,  local  minimizers,  and  stationary  points  of  the  discretized  problems  converge  to 
global  minimizers,  local  minimizers,  and  stationary  points,  respectively,  of  the  orig¬ 
inal  problems.  We  also  investigate  the  rate  of  convergence  of  algorithms  based  on 
discretization  schemes  as  a  computing  budget  tends  to  infinity.  We  provide  numer¬ 
ical  results  to  show  that  our  discretization  schemes  are  computationally  tractable, 
including  examples  with  three  searchers  and  ten  targets.  We  develop  three  heuris¬ 
tics  for  real-time  search  planning,  one  based  on  our  discretization  schemes,  and  two 
based  on  polynomial  fitting  methods,  and  compare  the  three  methods  to  determine 
which  solution  technique  would  be  best  suited  for  use  onboard  unmanned  platforms 
for  automatic  route  generation  for  search  missions. 
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EXECUTIVE  SUMMARY 


We  develop  formulations  for  continuous  time-and-space  search  problems  as  general¬ 
ized  optimal  control  problems,  where  searchers  look  for  non-evading  targets  that  move 
in  a  w-dimensional  area  of  interest,  where  w  is  a  positive  integer.  We  consider  a  large 
class  of  targets,  where  we  assume  the  targets  follow  deterministic  trajectories,  given 
some  information  about  their  initial  states  or  other  parameters.  We  deal  with  two 
categories  of  targets,  one  where  the  targets  coordinate  their  actions  and  one  where 
the  targets  operate  independently. 

We  refer  to  the  generalized  optimal  control  problem  when  the  targets  coordi¬ 
nate  their  actions  as  the  general  target  problem.  We  provide  two  different  problem 
formulations  for  this  category.  The  unconstrained  and  constrained  problems  ( GTP ) 
and  ( GTPC ),  respectively,  are  formulated  to  minimize  the  probability  that  all  of  the 
searchers  fail  to  detect  any  of  the  targets  during  the  planning  horizon.  The  uncon¬ 
strained  and  constrained  problems  ( GTPe )  and  ( GTPc,e ),  respectively,  are  formulated 
to  maximize  the  expected  number  of  targets  detected  during  the  planning  horizon. 

We  refer  to  the  generalized  optimal  control  problem  when  the  targets  operate 
independently  as  the  independent  target  problem,  and  formuate  the  unconstrained 
and  constrained  problems  ( ITPP )  and  ( ITPC'P ),  respectively,  to  minimize  the  prob¬ 
ability  that  all  of  the  searchers  fail  to  detect  any  of  the  targets  during  the  planning 
horizon.  We  also  formuate  the  unconstrained  and  constrained  problems  ( ITPe )  and 
( ITPc'e ),  respectively,  to  maximize  the  expected  number  of  targets  detected  during 
the  planning  horizon.  We  develop  discretization  schemes  to  solve  (GTP),  ( GTPC ), 
(< GTPe ),  (GTPc'e),  (. ITPp ),  (ITPC,P),  (. ITPe ),  and  (ITPc’e),  and  show  that  the  re¬ 
sulting  finite-dimensional  problems  are  consistent  approximations  to  their  infinite 
dimensional  counterparts.  Consistency  of  approximation  ensures  that  global  mini- 
rnizers,  local  minimizers,  and  stationary  points  of  the  discretized  problems  converge 


xv 


to  global  minimizers,  local  minimizers,  and  stationary  points,  respectively,  of  the 
original  problems. 

We  consider  a  broad  class  of  infinte  dimensional  optimization  problems,  which 
includes  ( GTP ),  ( GTPC ),  ( GTPe ),  (GTPc’e),  ( ITP p),  (ITPc’p),  (ITPe),  and  ( ITPc’e ). 
We  derive  rate  of  convergence  results  for  discretization  methods  used  to  solve  prob¬ 
lems  of  this  class  by  expressing  the  rate  of  convergence  in  terms  of  computational 
work.  We  find  an  upper  bound  for  the  rate  of  convergence  by  considering  a  finitely 
convergent  optimization  algorithm.  The  finitely  convergent  algorithm  has  no  opti¬ 
mization  error  after  a  sufficiently  large  number  of  iterations,  so  the  upper  bound  on 
the  rate  of  convergence  represents  the  best  possible  rate.  We  show  that  both  super- 
linear  and  linear  optimization  algorithms  are  also  able  to  attain  this  upper  bound, 
and  we  identify  specific  discretization  policies  that  achieve  this  best  possible  rate. 

We  provide  implementablc  algorithms  based  on  our  discretization  schemes 
along  with  numerical  results  to  show  that  the  problems  ( GTPC ),  ( ITPC,P ),  and 
(JTPc,e)  are  computationally  tractable.  For  the  90  cases  we  consider,  which  include 
examples  with  three  searchers  and  ten  targets,  we  are  able  to  compute  near-stationary 
solutions  in  the  range  of  3  to  20  hours. 

We  develop  three  heuristics  for  real-time  search  planning,  one  based  on  our 
discretization  schemes,  and  two  based  on  polynomial  fitting  methods,  and  compare 
the  three  methods  to  determine  which  solution  technique  would  be  best  suited  for  use 
onboard  unmanned  platforms.  Our  numerical  results  indicate  that  our  fixed-precision 
discretization  scheme  consistently  provides  solutions  with  the  best  objective  values 
in  the  range  of  40  to  200  seconds,  and  is  therefore  the  best  candidate  for  use  as  a 
real-time  search  planning  method. 
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I.  INTRODUCTION 

A.  MOTIVATION  AND  BACKGROUND 

The  need  to  protect  a  High  Value  Unit  (HVU),  such  as  an  aircraft  carrier 
or  other  large  capital  ship,  from  small  boat  attack  is  an  important  problem  faced 
by  navies  around  the  world.  The  need  to  protect  HVUs  from  these  types  of  attack 
was  brought  into  sharp  focus  following  the  attack  on  the  USS  Cole  (DDG  67)  in 
2000.  The  need  was  further  demonstrated  during  a  U.S.  Navy  war  game  in  2002, 
in  which  adversarial  forces  used  swarms  of  small  boats  and  aircraft  to  overwhelm  a 
U.S.  invasion  fleet  (Kahwaji,  2006).  There  are  also  indications  that  smaller  countries 
could  resort  to  an  asymmetric  strategy  involving  small  boats  if  they  were  involved 
in  a  naval  conflict  against  a  major  naval  power  (Kahwaji,  2006).  A  key  component 
of  any  defense  against  small  boat  attack  involves  the  search  for  and  detection  of 
potential  adversaries.  To  quote  Kahwaji,  “Surveillance  is  key:  If  the  raiders  can  be 
tracked  as  they  swarm  from  their  bases,  they  can  be  sunk  with  Rockeye  cluster  bombs 
and  other  munitions”  (Kahwaji,  2006).  In  order  to  provide  the  best  possible  defense 
against  small  boat  attack,  it  is  clear  that  optimal  utilization  of  search  assets  is  highly 
desirable. 

Because  the  searchers’  trajectories  have  a  significant  impact  on  the  probability 
of  hireling  the  targets  within  a  given  time  horizon,  we  would  like  a  way  to  determine 
the  “best”  trajectory  for  each  of  the  searchers.  This  is  a  fundamentally  difficult 
problem  due  to  the  nonlinearity  and  nonconvexity  introduced  by  using  probability  of 
detection  as  the  basic  performance  measure  of  a  search  platform.  It  becomes  even 
more  difficult  if  we  consider  multiple  searchers  looking  for  multiple  targets.  Given  the 
increasing  use  of  unmanned  systems,  such  as  unmanned  aircraft  systems  (UASs)  and 
unmanned  surface  vehicles  (USVs),  by  militaries  around  the  world,  it  is  clear  that 
there  is  a  need  for  an  automated  method  for  finding  optimal  search  trajectories  given 
some  initial  intelligence  information  about  potential  adversaries.  This  need  serves  as 
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motivation  for  our  work.  In  this  dissertation,  we  consider  several  search  trajectory 
optimization  problems,  including  cases  with  multiple  searchers  looking  for  multiple 
targets.  The  searchers  could  be  aircraft  or  surface  vessels  with  sensors  designed  to 
detect  targets  that  pose  a  threat  to  a  HVU.  We  formulate  these  search  problems  as 
generalized  optimal  control  problems,  in  continuous  time  and  space.  We  develop  and 
analyze  algorithms  based  on  discretization  schemes  to  solve  these  problems;  these 
schemes  are  consistent  approximations  in  the  sense  of  Polak  (1997),  and  the  resulting 
algorithms  are  guaranteed  to  converge  to  stationary  solutions. 

B.  SCOPE  OF  DISSERTATION 

We  consider  search  trajectory  optimization  problems  where  searchers  look  for 
nonevading  targets  that  move  in  a  w-dimensional  area  of  interest,  where  w  is  a  positive 
integer.  We  limit  the  scope  to  searchers  that  move  in  continuous  time  and  space, 
according  to  dynamics  defined  by  ordinary  differential  equations.  It  is  worth  noting 
that  this  includes  a  large  class  of  searchers,  such  as  many  manned  and  unmanned 
aircraft  as  well  as  many  manned  and  unmanned  surface  vessels.  We  consider  a  large 
class  of  targets  whose  trajectories  may  not  be  continuous  in  time  and  space.  While  we 
only  consider  non-evading  targets,  targets  are  allowed  to  move  intelligently.  We  allow 
for  the  case  of  targets  that  have  perfect  knowledge  of  the  HVU’s  position  at  all  time, 
who  then  determine  the  trajectories  required  to  strike  the  HVU  in  minimum  time.  We 
require  that  the  targets’  motion  is  conditionally  deterministic,  which  means  that  the 
targets  follow  deterministic  trajectories  given  realizations  of  random  variables  that 
specify  information  about  their  initial  states  or  other  parameters.  We  assume  that 
the  probability  distribution  for  each  random  variable  is  known,  and  we  consider  two 
situations.  The  first  is  when  the  random  variables  are  dependent.  This  would  be  true, 
for  example,  if  the  targets  chose  to  attack  cooperatively  in  a  swarm  configuration. 
The  second  is  when  the  random  variables  describing  one  target  are  independent  of 
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the  random  variables  describing  another  target.  This  would  represent  the  case  when 
the  targets  attack  with  no  level  of  coordination,  other  than  the  desire  to  strike  the 
same  HVU. 

We  refer  to  the  generalized  optimal  control  problem  when  the  targets  coordi¬ 
nate  their  actions  as  the  general  target  problem.  We  provide  two  different  problem 
formulations  for  the  case  when  the  targets  coordinate  their  actions.  The  first  formu¬ 
lation  has  corresponding  problems  ( GTP )  and  ( GTPC ),  for  the  unconstrained  and 
control-constrained  cases,  respectively,  whose  solutions  minimize  the  probability  that 
all  of  the  searchers  fail  to  detect  any  of  the  targets  during  the  planning  horizon.  The 
second  formulation  has  corresponding  problems  ( GTPe )  and  ( GTPc,e ),  for  the  un¬ 
constrained  and  control-constrained  cases,  respectively,  whose  solutions  maximize  the 
expected  number  of  targets  detected  during  the  planning  horizon. 

We  refer  to  the  generalized  optimal  control  problem  when  the  random  variables 
are  independent  across  targets  as  the  independent  target  problem.  We  also  provide 
two  different  problem  formulations  for  the  independent  target  category.  The  first 
formulation  has  corresponding  problems  ( ITPP )  and  ( ITPC'P ),  for  the  unconstrained 
and  control-constrained  cases,  respectively,  whose  solutions  minimize  the  probability 
that  all  of  the  searchers  fail  to  detect  any  of  the  targets  during  the  planning  horizon. 
The  second  formulation  has  corresponding  problems  ( ITPe )  and  ( ITPc,e ),  for  the 
unconstrained  and  control-constrained  cases,  respectively,  whose  solutions  maximize 
the  expected  number  of  targets  detected  during  the  planning  horizon. 

Previous  work  on  solving  search  problems  using  optimal  control  has  focused 
on  deriving  necessary  conditions  for  optimality  (see,  for  example,  Heilman,  1970, 
1971,  1972,  and  Lukka,  1977)  in  the  tradition  of  Pontryagin,  or  sufficient  conditions 
for  optimality  (see,  for  example,  Hibey,  1982  and  Ohsumi,  1991)  in  the  tradition  of 
Hamilton,  Jacobi,  and  Bellman.  Because  it  is  unclear  how  to  use  these  approaches  to 
develop  consistent  approximations  that  converge  to  solutions  of  the  original  problem, 
we  adopt  the  method  developed  by  E.  Polak  (see,  for  example,  Section  4.2  in  Polak, 
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1997)  to  define  optimality  conditions.  As  in  Section  4.3  of  Polak  (1997),  we  use  Euler’s 
method  to  approximately  solve  the  time-discretized  ordinary  differential  equations 
governing  the  searcher  dynamics.  While  it  is  possible  to  extend  Polak’s  method 
to  include  Runge-Kutta  integration  methods  (see,  for  example,  Schwartz  &  Polak, 
1996),  doing  so  introduces  additional  complications  in  both  theory  and  numerical 
implementation.  Because  this  is  the  first  attempt  to  utilize  Polak’s  method  to  derive 
optimality  conditions  and  solutions  for  these  types  of  generalized  optimal  control 
problems,  we  use  Euler’s  method  throughout  this  dissertation  to  approximately  solve 
the  time-discretized  ordinary  differential  equations. 

The  problems  we  have  formulated  require  spatial  integration  in  addition  to  in¬ 
tegration  in  time.  Spatial  integration  often  requires  more  than  one  dimension,  making 
numerical  methods  used  to  approximate  spatial  integrals  more  computationally  ex¬ 
pensive  than  those  related  to  time  integration.  We  use  Simpson’s  integration  rule 
as  a  higher-order  approximation  to  the  spatial  integrals  due  to  the  fact  that  it  helps 
limit  approximation  error,  and  is  relatively  simple  to  handle  in  both  theory  as  well 
as  implementation. 

C.  LITERATURE  SURVEY 

Modern  search  theory  traces  its  roots  to  the  formation  of  the  United  States 
Navy’s  Operations  Research  Group  (ORG)  during  World  War  II.  During  the  war, 
Dr.  B.  O.  Koopman  and  his  associates  in  the  ORG  worked  on  improving  the  way  the 
United  States  Navy  conducted  search  during  anti-submarine  warfare  operations  (Iida 
et  ah,  2002).  Koopman  (1980)  is  an  unclassified  and  updated  version  of  Koopman 
(1946),  which  summarizes  the  work  done  by  the  ORG  during  the  war.  Koopman 
(1980),  and  indeed  much  of  the  literature  prior  to  the  1970s,  focuses  on  stationary 
targets  (Benkoski  et  ah,  1991).  Research  literature  on  moving  targets  can  be  grouped 
into  two  categories  as  given  in  Benkoski  et  al.  (1991): 


4 


(i)  articles  which  address  special  types  of  target  motion,  described  below,  that  are 
amenable  to  analysis  and 

(ii)  articles  which  build  on  S.  Brown’s  work  on  conditioning  with  stationary  tar¬ 
gets  (see  Brown,  1980)  by  developing  general  necessary  and  sufficient  optimality 
conditions  for  moving-target  problems. 

In  this  dissertation,  we  deal  with  target  motion  that  is  amenable  to  analysis,  so  we 
refer  the  interested  reader  to  Benkoski  et  al.  (1991)  for  a  comprehensive  survey  of  the 
literature  related  to  category  (ii). 

With  respect  to  category  (i),  there  are  two  special  types  of  target  motion  that 
appear  in  the  literature.  The  first  type  deals  with  targets  whose  motion  is  Marko¬ 
vian  in  nature,  which  means  that  the  target  moves  in  a  random  manner  that  can  be 
modeled  by  a  Markov  process.  Benkoski  et  al.  (1991)  again  provides  a  comprehen¬ 
sive  literature  survey  for  Markovian  target  motion  studies  (until  1990).  More  recent 
research  (see,  for  example,  Washburn,  1998;  Lau,  Huang,  &  Dissanayake,  2008,  and 
Sato  &  Royset,  2010)  focuses  on  the  development  of  specialized  branch- and-bound 
algorithms  for  hireling  an  optimal  path  for  the  single  searcher.  In  addition,  Dell  et  al. 
(1996)  present  an  exact  procedure  (utilizing  a  branch-and-bound  algorithm)  as  well 
as  six  heuristics  (local  search,  expected  detection  heuristics,  genetic  algorithms,  and 
moving  time-horizon  heuristic)  for  solving  the  multiple  searcher  problem. 

The  second  type  of  special  target  motion  in  the  literature  is  when  the  tar¬ 
get’s  motion  is  assumed  to  be  conditionally  deterministic.  The  term  conditionally 
deterministic  means  that  the  target’s  trajectory  depends  on  random  variables,  and 
if  the  random  variables  are  given,  then  the  target’s  position  is  known  for  all  time. 
This  type  of  target  motion  has  been  investigated  by  many  different  researchers.  In 
the  case  of  discrete  time,  Royset  and  Sato  (2010)  presents  a  convex  mixed-integer 
nonlinear  program  (MINLP)  formulation  for  a  route  optimization  problem  involving 
multiple  searchers  who  seek  to  detect  one  or  more  moving  targets.  Royset  and  Sato 
(2010)  propose  two  solution  approaches  for  the  MINLP  based  on  linearizations,  one 
of  which  involves  using  a  cutting-plane  method. 
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For  the  case  of  continuous  time,  Stone  and  Richardson  (1974)  and  Stone  (1977) 
derive  necessary  and  sufficient  conditions  to  optimally  allocate  search  effort  to  maxi¬ 
mize  the  probability  of  detection  during  a  given  planning  horizon,  lida  (1989)  extends 
the  work  of  Stone  and  Richardson  to  include  the  problem  of  finding  the  optimal  search 
plan  which  minimizes  the  expected  risk  (the  expected  search  cost  minus  the  expected 
reward),  lida  (1989)  also  derives  the  closed  form  of  the  optimal  search  plan  when  the 
target  moves  straight  from  a  fixed  point  and  selects  its  course  and  speed  randomly. 
Pursiheimo  (1976)  derives  a  necessary  condition  for  search  plans  to  be  optimal  when 
the  probability  of  detection  is  to  be  maximized  and  expected  search  time  is  to  be 
minimized  for  a  continuous  time,  discrete  space  model.  The  approach  in  Pursiheimo 
(1976)  is  noteworthy  since  it  is  the  first  to  formulate  the  search  problem  for  a  target 
with  conditionally  deterministic  motion  as  an  optimal  control  model.  The  results  in 
Pursiheimo  (1976)  are  theoretical  in  nature,  however,  as  no  search  plans  are  gener¬ 
ated. 

Following  the  work  of  Pursiheimo,  there  have  been  other  formulations  of  the 
search  problem  using  an  optimal  control  model.  Lukka  (1977)  uses  an  optimal  con¬ 
trol  model  and  derives  a  necessary  condition  for  the  optimal  search  plan  assuming 
conditionally  deterministic  target  motion.  Mangel  (1981)  presents  an  approximate 
solution  method,  referred  to  as  the  “ray  method,”  that  can  be  used  in  conjunction 
with  the  necessary  conditions  given  by  Lukka  (1977)  to  determine  the  optimal  search 
plan.  Hibey  (1982)  and  Ohsumi  (1991)  use  an  optimal  control  model  to  find  sufficient 
conditions  for  the  optimal  search  plan  assuming  Markovian  target  motion.  Ohsumi 
(1991)  also  gives  a  method  that  can  be  used  to  numerically  approximate  the  optimal 
search  trajectories,  with  simulation  studies  assuming  Markovian  target  motion. 

It  should  be  noted  that  Lukka  (1977),  Mangel  (1981),  Hibey  (1982),  and 
Ohsumi  (1991)  all  use  an  indirect  approach  to  solving  the  optimal  control  prob¬ 
lem,  employing  the  calculus  of  variations  to  obtain  first-order  optimality  conditions, 
which  result  in  a  boundary-value  problem  that  must  be  solved.  A  direct  method 
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can  also  be  used  to  solve  the  optimal  control  problem,  which  eliminates  the  need  to 
solve  a  boundary-value  problem.  In  a  direct  method,  a  time-discretization  scheme  is 
introduced,  and  the  control  problem  is  then  transcribed  into  a  nonlinear  optimization 
problem  that  can  be  solved  using  standard  techniques.  Wasburn  (1990)  employs  a 
direct  method  to  construct  a  tactical  decision  program,  JITTER,  which  is  designed  to 
find  an  optimized  trajectory  for  a  submarine  transiting  from  a  given  starting  location 
to  a  given  terminal  location,  while  minimizing  the  total  acoustical  energy  received  by 
n  listeners.  JITTER  utilizes  discretization  and  a  steepest-descent  method  that  grad¬ 
ually  modifies  a  user-supplied  initial  trajectory  into  an  optimized  path  by  making 
first-order  corrections. 

A  more  recent  approach,  given  in  Chung  et  al.  (2010),  uses  a  direct  method 
based  on  Chapter  4  of  Polak  (1997)  to  solve  an  optimal  control  problem  which  Ends 
optimized  search  plans  for  a  target  that  moves  straight  down  a  channel  at  constant 
speed.  The  results  in  Chung  et  al.  (2010)  indicate  that  the  direct  method  based  on 
Chapter  4  of  Polak  (1997)  can  be  used  to  generate  optimized  search  plans  for  as  many 
as  three  searchers.  Because  the  direct  method  based  on  Chapter  4  of  Polak  (1997) 
demonstrates  the  ability  to  produce  search  plans  and  allows  us  to  develop  consistent 
approximations,  we  focus  on  this  method  in  the  dissertation. 

D.  CONTRIBUTIONS 

In  this  dissertation,  we  extend  the  work  of  Chung  et  al.  (2010)  in  many  ways. 
Our  target  motion  model  is  more  general  than  that  found  in  Chung  et  al.  (2010),  and 
as  a  result  the  framework  we  develop  allows  for  the  formulation  of  many  important 
continuous  time-and-space  search  problems  as  generalized  optimal  control  problems. 
We  provide  four  different  examples  of  problem  formulations,  namely  ( GTP ),  ( GTPe ), 
( ITPP ),  and  (JTPe),  that  can  be  modeled  using  our  framework.  We  also  develop  a 
min-max  formulation  that  can  be  used  in  future  studies  to  find  searcher  trajectories 
that  minimize  the  maximum  expected  time  of  first  detection  of  all  the  potential 
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targets,  as  well  as  a  model  that  can  extend  our  detection-based  approach  to  one  that 
includes  herding  potential  threats  away  from  a  high  value  unit.  As  in  Chung  et  al. 
(2010),  we  also  use  a  direct  method  to  solve  the  generalized  optimal  control  problems 
based  on  Chapter  4  of  Polak  (1997),  but  we  are  the  first  to  provide  proofs  to  show  that 
the  discretization  schemes  are  consistent  approximations  to  their  infinite-dimensional 
counterparts. 

We  develop  rate  of  convergence  results  based  on  expressing  the  rate  of  conver¬ 
gence  in  terms  of  computational  work  rather  than  the  traditional  number  of  iterations 
or  level  of  discretization.  Using  this  approach,  we  provide  an  upper  bound  on  the 
rate  of  convergence  that  can  be  achieved  by  any  optimization  algorithm.  We  also 
provide  discretization  policies  for  superlinearly  and  linearly  convergent  optimization 
algorithms  that  achieve  this  upper  bound.  In  addition,  we  use  the  rate  of  convergence 
results  we  obtain  to  provide  insight  regarding  the  choice  of  numerical  method  used 
to  approximately  solve  the  differential  equations  as  well  as  approximate  the  spatial 
integration  when  solving  generalized  optimal  control  problems. 

We  provide  implementable  algorithms  based  on  our  discretization  schemes, 
and  use  them  to  produce  numerical  results  to  show  that  our  method  is  computation¬ 
ally  tractable.  While  Chung  et  al.  (2010)  considers  a  single  target  and  as  many  as 
three  searchers,  we  give  numerical  examples  that  include  three  searchers  looking  for 
as  many  as  ten  targets.  We  also  investigate  methods  to  reduce  the  computational 
cost  necessary  to  obtain  these  numerical  solutions,  including  adaptive  discretization 
schemes  and  heuristics  based  on  polynomial  fitting. 

E.  ORGANIZATION 

The  remainder  of  this  dissertation  is  outlined  as  follows.  Chapter  II  gives  the 
objective  function  definitions,  describes  the  target  motion  model,  defines  the  searcher 
dynamics,  and  provides  preliminary  generalized  optimal  control  problem  formulations. 
Chapter  III  introduces  the  spaces  necessary  to  complete  the  problem  formulations  and 


develops  consistent  approximations  for  the  generalized  optimal  control  problems  we 
consider  in  the  dissertation.  In  Chapter  IV,  we  develop  rate-of-convergence  results 
for  different  classes  of  optimization  algorithms  that  can  be  used  to  solve  generalized 
optimal  control  problems.  Chapter  V  provides  implementable  algorithms  and  gives 
our  numerical  results.  In  Chapter  VI  we  present  conclusions  and  suggest  future 
research  opportunities.  The  Appendix  provides  some  mathematical  background. 
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II.  SITUATIONAL  DESCRIPTION  AND 
PROBLEM  FORMULATIONS 

We  consider  the  situation  depicted  in  Figure  1,  where  a  High  Value  Unit 
(HVU)  is  operating  in  a  two-dimensional  area  of  interest.  The  HVU  follows  a  fixed 
trajectory,  {x°(t)  G  M2  :  0  <t  <T  <  oo},  during  the  finite  planning  horizon  [0,  T\. 
Without  loss  of  generality,  we  assume  throughout  the  dissertation  that  T  =  1,  and 
therefore  our  normalized  planning  horizon  is  [0,1].  During  the  transit  from  x°(0) 
to  rr0(l),  the  HVU  is  under  threat  of  attack  from  L  targets.  We  assume  that  the 
motion  of  these  targets  is  conditionally  deterministic,  i.e.,  the  Ith  target’s  position, 
yl{t\a ),  is  known  for  all  time  t  e  [0,1]  given  the  realization  of  a  random  vector  a 
which  takes  values  in  a  compact  set  A  C  where  w  is  a  positive  integer.  It  should 
be  noted  that  a  will  be  used  to  represent  both  a  random  vector  and  its  realization, 
but  the  nature  of  a  should  be  clear  from  the  context.  We  assume  that  the  probability 
distribution  for  a  is  known.  In  an  effort  to  detect  the  potential  threats  to  the  HVU, 
there  are  K  searchers  operating  in  the  vicinity  of  the  HVU.  Our  goal  is  to  determine 
the  trajectory,  xk(t),  for  each  of  the  searchers  to  follow  during  the  planning  horizon 
that  optimizes  objective  functions  we  define  in  Sections  II. A  and  II. B. 

There  are  many  different  ways  to  measure  the  effectiveness  of  a  proposed 
search  plan.  In  this  chapter,  and  throughout  the  dissertation,  we  focus  on  two  types 
of  objective  functions.  The  theory  we  develop  can  be  extended  to  include  other  types 
of  objective  functions  (see,  for  example,  Chapter  VI,  Sections  B.l  and  B.2),  but  for 
our  analysis  and  numerical  results  we  concentrate  on  two  types  of  objective  functions: 
the  first  type  seeks  to  minimize  the  probability  that  all  of  the  searchers  fail  to  detect 
any  of  the  targets  during  the  planning  horizon;  the  second  type  seeks  to  maximize 
the  expected  number  of  distinct  targets  detected  during  the  planning  horizon. 

In  this  chapter,  we  begin  by  developing  both  types  of  objective  functions  for 
the  case  of  statistically  dependent  targets.  The  term  dependent  targets  refers  to  the 
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case  when  the  random  variables  that  define  the  targets’  motion  are  dependent  among 
targets.  A  coordinated  swarm  attack  against  the  HVU  would  be  appropriately  mod¬ 
eled  by  the  dependent  target  case.  We  then  derive  both  types  of  objective  functions 
for  the  case  of  independent  targets.  The  term  independent  targets  refers  to  the  case 
when  the  random  variables  that  define  the  targets’  motion  are  independent  across 
targets.  It  should  be  noted  that  in  the  case  of  independent  targets  there  may  be 
dependence  between  the  random  variables  that  specify  information  about  the  pa¬ 
rameters  for  a  particular  target.  An  attack  by  multiple  targets  whose  only  level  of 
coordination  is  the  desire  to  strike  the  same  HVU  would  be  appropriately  modeled 
by  the  independent  target  case.  Next,  we  discuss  the  target  motion  model.  Finally, 
we  give  preliminary  definitions  of  the  problems  we  consider  in  this  dissertation.  We 
complete  these  problem  definitions  in  Chapter  III,  after  we  define  the  appropriate 
spaces  for  the  searcher  control  inputs. 
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A.  COORDINATED  TARGETS 

Given  a  G  A  C  l1"  common  to  all  targets,  the  Ith  target  follows  a  deterministic 
trajectory,  {yl(t\  a)  G  Mny  :  0  <  t  <  l},  where  ny  is  a  positive  integer.  As  an  example, 
a  could  be  used  to  represent  the  uncertainty  in  target  starting  location  and  starting 
time.  Then,  given  the  starting  location  and  starting  time  for  the  Ith  target,  some 
type  of  deterministic  algorithm  can  be  used  to  find  the  trajectory  for  the  Ith  target. 
While  the  formulation  that  follows  is  valid  for  any  w,  the  dimension  of  A,  it  appears 
difficult  to  carry  out  the  analysis  for  an  arbitrary  w.  This  is  because  w  influences 
the  choice  and  characteristics  of  the  numerical  integration  scheme  used  to  calculate 
search  plans.  During  later  proofs  of  convergence  it  is  necessary  to  quantify  the  error 
introduced  by  the  numerical  integration  scheme,  so  we  now  fix  w  =  2. 

We  let  rk’1  :  M”  x  Mny  — »  [0,oo),  were  n  is  a  positive  integer,  denote  the 
detection  rate  for  searcher  k  against  target  /,  which  is  defined  such  that  rk,l(xk,yl)At 
approximates  the  probability  of  the  kth  searcher  in  state  xk  G  Mn  detecting  the  Ith 
target  in  state  yl  G  Mny  during  a  small  time  interval  [t,t  +  At).  The  states  for  the 
searcher  and  target  typically  involve  their  locations,  but  could  also  include  other 
quantities  such  as  heading  and  time  of  day.  The  detection  rate  reflects  the  sensor 
effectiveness  and  we  typically  have  that  rk,l(xk:yl )  is  some  decreasing  function  in 
the  “distance”  between  xk  and  yl.  We  put  distance  in  quotations  because  when 
n  ^  ny.  it  is  necessary  to  omit  certain  portions  of  the  state  vectors  for  xk  and  yl  in 
order  to  properly  define  a  norm  that  provides  a  measure  of  the  closeness  of  the  two 
state  vectors.  The  detection  rate,  rk,t (•,•),  can  be  selected  to  reflect  various  types 
of  sensors  and  their  performance  against  different  types  of  targets.  For  theoretical 
and  computational  reasons,  rk,l(-,  ■)  must  satisfy  certain  differentiability  assumptions, 
which  we  state  in  Assumption  111.3. 

Next,  we  derive  the  detection  model  for  a  particular  a,  but  first  we  define  some 
notation.  In  a  manner  similar  to  Chung  et  al.  (2010),  given  a  particular  trajectory 
for  the  kth  searcher  {xk{t)  :  0  <  t  <  1}  and  a  particular  trajectory  for  the  Ith  target 
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{yl(t;a)  :  0  <  t  <  1}  under  realization  a,  we  denote  the  probability  that  the  kth 
searcher  does  not  detect  the  Ith  target  during  [0,  t],  t  e  [0, 1],  by  qk,l(t]  a).  We  assume 
that  events  of  detection  in  non-overlapping  time  intervals  are  all  independent,  so  we 
can  calculate  qk,l(t;  a)  recursively  using  the  difference  equation1 

qk,l(t  +  At]  a)  =  qk'\t\  a)  (l  -  (rk’\xk(t),  y\t\  a)) At  +  o(Af)))  ,qk,l(0]a)  =  1,  (11.1) 


which  becomes  the  parameterized  differential  equation 


qk-‘(t]a)  4  =  -qk-l(t]a)rk-l(x\t),yl(t]a)),  qkJ(0;a)  =  1,  (11,2) 


as  At  — )■  0,  with  solution 


qk'\t-a )  =  exp  (  —  /  rk’l(xk(s),  y\s-  a))ds  )  . 

Jo 


(II.3) 


We  assume  that  the  searchers  make  independent  detection  attempts  and  can 
simultaneously  detect  multiple  targets,  and  hence  it  follows  from  (11.3)  that  the  con¬ 
ditional  probability  that  no  searcher  detects  any  target  during  the  time  period  [0, 1], 
given  a  and  collection  of  searcher  trajectories,  {xk{t)  :  0  <  t  <  1},  k  —  1,  2,  is 

simply  the  product 


kl,  i  \  /  k  L  ! 

1 1 II  exP  (  ~  /  rk’l(xk(t),  y\t\  a))dt )  =  exp  -  ^  ^  \ 
k= i  i=i  \  Jo  /  \  k=1  l=1  Jo 

=  exp  -  /  A  A 
V  J°  k= 1 1=1 


rk’l(xk(t),yl(t-,  a))dt 
rk,t (xk {t) ,  yl (t;  a))dt 


Similarly,  the  conditional  probability  that  no  searcher  detects  the  Ith  target 
during  the  time  period  [0,1],  given  a  and  collection  of  searcher  trajectories,  { xk(t )  : 
0  <  t  <  1},  k  =  1,  2, K,  is  given  by  the  product 


K 

II exp 


k=  1 


"1  \  /  r  1  K 

„k,l  t „k  „.l/ 


rk,l(xk(t),yL(t]a))dtJ  =  exp  I  —  J  ^  rk,l(xk(t),  yl(t;  a))dt  j  . 


1  Recall  that  if  a  function  /  : 
on  page  304  in  Ross  (2007). 


k= 1 


is  o( x),  then  liniaj^o  =  0;  see  for  example  Definition  5.2 
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The  conditional  probability  that  at  least  one  of  the  searchers  detects  the  Ith  target 
during  the  planning  horizon  [0,1],  given  a  and  collection  of  searcher  trajectories, 
{xk(t)  :  0  <  t  <  1},  k  —  1,  2, ...,  K,  is  given  by 


ri  k 

1  -  exp  (  -  /  rk'\xk(t),y\t-,  a))dt  ]  . 

J°  /■'=  1 


(11.4) 


Then,  the  expected  number  of  targets  detected  during  the  time  period  [0, 1],  given  a 
and  collection  of  searcher  trajectories,  {xk(t)  :  0  <  t  <  1},  k  —  1,  2, K,  is  given  by 


E 

i=i 


-1  K 


1  -  exp  -  /  y ^rk'l(xk(t),yl(t-,a))dt 


k= 1 


(11,5) 


We  let  (j)  :  A  — >  M  be  the  probability  density  function  of  a.  For  theoretical  and 
computational  reasons,  0(-)  must  satisfy  certain  differentiability  assumptions  which 
we  state  in  Assumption  111.1.  Then,  the  probability  that  all  of  the  searchers  fail  to 
detect  any  of  the  targets  during  [0, 1]  is  given  by 


1  I<  L 

exp  (  —  /  YYrk,l(xk(t)^yl(t'^a))dt  )  (p(oi)da, 

Jo  k= i  i=i 


(IL6) 


and  the  expected  number  of  targets  detected  during  the  time  period  [0, 1]  is  given  by 


E 


'“eA  «=i 


„1  K 


1  —  exp  [—  J 


E 

(=i 


k=l 
1  K 


(j)(a)da 


1  - 


'  a£A 


exp  J  rk,l(xk(t),yl(t]a))dtj  4>(a)da 


(11.7) 


B.  INDEPENDENT  TARGETS 

We  now  assume  that  the  random  variables  that  the  target  motion  is  con¬ 
ditioned  upon  are  independent  among  targets.  We  then  have  a  vector  of  random 
variables  a1  e  A  C  M>w,  one  for  every  target,  and  we  assume  that  the  probabil¬ 
ity  distribution  for  a1  is  known  for  all  l  =  1,2,  Given  a  particular  realiza¬ 

tion  of  the  random  vector,  a1,  then  the  Ith  target  follows  a  deterministic  trajectory, 
{yl(t\  a1)  G  Wly  :  0  <  t  <  l}.  We  note  that  the  components  of  a1  can  be  dependent. 
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Following  the  same  development  as  (II. 1)  through  (II. 3),  with  a  replaced  by 
a1,  we  find  that  the  conditional  probability  that  the  kth  searcher  fails  to  detect  the 
Ith  target  during  the  time  period  [0, 1],  given  a1  and  searcher  trajectory  { xk(t )  :  0  < 
t  <  1},  is  given  by 

exp  rk’l(xk(t),yl(t-al))d?j  .  (II.8) 

We  assume  that  the  searchers  make  independent  detection  attempts,  and  hence  it 
follows  from  (II. 8)  that  the  conditional  probability  that  no  searcher  detects  the  Ith 
target,  given  a1  and  collection  of  searcher  trajectories,  {xk{t)  :  0  <  t  <  1},  k  = 
1,  2, ...,  K,  is  simply  the  product 

TT  exp  f  —  f  rk’l(xk(t),y\t-al))dt]  =  expf-V/  rk’l{xk(t),y\t;  al))dt 

k= i  \  '  V  k=iJo 

(  r1  K  \ 

■/o  k=  i  1 


=  exp 


We  let  (j)1  :  A  — »  M  be  the  probability  density  function  of  a\  l  =  1,2 , ...,  L,  and 
require  it  to  satisfy  certain  differentiability  assumptions,  which  we  state  in  Assump¬ 
tion  III. 31.  Then,  the  probability  that  all  of  the  searchers  fail  to  detect  the  Ith  target 
during  the  time  period  [0, 1]  is  given  by 

f  y^rk,l(xk(t),yl(t;al))dt  \  <f>l(al)dal.  (II. 9) 

/o  k= i  J 

Hence,  the  probability  that  all  of  the  searchers  fail  to  detect  any  of  the  targets  during 
[0, 1]  is  given  by 


I  aleA 


exp 


L 

n 

i=i 


/  a‘eA 


exp 


—  f  y ^yrk’l(xk(t),yl(t-,  al))dt  \  <j)l(al)dal 

Jo  k= i  1 


(11.10) 


Based  on  (II. 9),  the  probability  that  at  least  one  of  the  searchers  detects  the  Ith  target 
during  the  planning  horizon  [0, 1]  is  given  by 

'  rl  K  \ 

—  j  yy  rk,l(xk(t),yl(t]  al))dt  J  (f)\al)dal.  (II. 11) 


exp 


I  aleA 
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Then,  the  expected  number  of  targets  detected  during  the  time  period  [0, 1]  is  given 
by 

exp  -  / 
eA  y  Jo 

We  use  (II. 6),  (II. 7),  (II. 10),  and  (11.12)  to  formulate  the  generalized  optimal 
control  problems  that  we  consider  in  this  dissertation,  but  first  we  discuss  the  target 
motion  model  which  we  use  to  generate  the  target  trajectories  {yl{t\  •)  G  :  0  < 
t  <  1}. 

C.  TARGET  MOTION  MODEL 

We  develop  this  section  for  the  coordinated  target  case,  but  note  that  the  inde¬ 
pendent  target  case  is  identical  except  that  a  is  replaced  by  a1.  Given  a  particular  a, 
the  target  trajectories,  {yl(t]  a)  G  Mny  :  0  <  t  <  1},  l  —  1,  2, L,  could  be  generated 
in  numerous  ways.  In  Chapter  III,  we  make  very  light  assumptions  regarding  ?/(•;  •), 
so  the  theory  we  develop  applies  to  a  broad  class  of  targets.  In  order  to  provide  a 
specific  numerical  example,  we  make  the  conservative  assumption  that  the  targets 
have  full  knowledge  of  the  position  of  the  HVU  at  all  time.  Based  on  this  information 
about  the  HVU,  we  model  the  targets  as  Dubins  vehicles,  i.e.,  nonholonomic  vehicles 
that  are  constrained  to  move  along  planar  paths  of  bounded  curvature,  without  re¬ 
versing  direction,  that  act  intelligently  and  follow  trajectories  that  seek  to  minimize 
the  time  required,  tf,  for  each  of  them  to  hit  the  HVU.  The  motion  of  the  targets  is 
subject  to  additional  constraints,  which  we  now  discuss  in  detail. 

For  any  t  G  [0,  tf\,  let  ny  =  3  and  yl(t;a)  =  (y[(t',a),yl2(t',a),yl3(t',a))T  G  M3, 
where  T  denotes  the  transpose  of  a  vector,  be  the  state  of  the  Ith  target  at  time  t,  with 
y[(t)  «)gR  and  yl2(t',  a)  G  M  denoting  the  horizontal  and  vertical  components  of  the 
location  of  the  Ith  target,  respectively,  and  yl3(t',a)  G  M  denoting  the  heading  of  the 
Ith  target  measured  from  the  horizontal  axis  at  time  t.  For  any  t  G  [0,  tf],  the  control 
input,  ul,tar(t),  for  the  target  is  the  rate  of  change  of  the  heading,  which  is  restricted 


K 

V 

k= 1 


rk,l(xk(t),yl(t;al))dt  ]  4>\al)dal 


(11.12) 
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to  the  range  [— ul,tar,  ul,tar].  The  target’s  speed,  \\(y[(t;a),yl2(t',a))\\,  is  restricted  to 
the  range  [vlmin,  vlmax\.  We  require  that  the  target’s  initial  and  final  speeds,  vl0  and  vlf, 
be  specified.  Recall  that  a  is  based  on  a  distribution,  then  given  a  particular  a,  the  Ith 
target’s  starting  position,  yl0(a),  and  starting  time,  tl0(a),  are  known  and  we  generate 
its  trajectory  by  approximately  solving  the  following  optimal  control  problem: 

min  t  f 

s.t.  y[(t-,a)  =  \\(y[(t;a),yl2(t-,a))\\  cos yl3(t;  a),  Vt  G  [tl0(a),tf]  (11.13) 

=  \\(y[(t]  a),  yl2(t]a))\\  sin yl3(t;  a),  Vt  e  [tl0(a),tf}  (11.14) 


yl3(t;a)  =  ul’tar(t),Vte[tl0(a),tf]  (11.15) 

Vlmin  <  Il(^(^;«)^2^;«))ll  <  Vt  G  [^ («),£/]  (11.16) 

-Ul’tar  <  Ul’tar(t)  <ul’tar,  Vt  e[tl0(a),tf]  (11.17) 

vl0  =  ||(^(4(«);«)^2(4(«);a))ll  (IL18) 

vlf  =  \\(y[(tf,a),yl2(tf,a))\\  (11.19) 

yl(tl0(a);a)  =  yl0(a )  (11.20) 

(■ x°i(tf),x°2(tf ))  =  (y[(tf,a),yl2{tf,a))  (11.21) 


The  target  dynamics  are  given  by  equations  (11.13),  (11.14),  and  (11.15).  Constraint 
(11.16)  restricts  the  target’s  speed  to  the  range  [vlmin,  vlmax] .  Constraint  (11.17)  restricts 
the  target’s  turn  rate  to  the  range  [— ul,tar,  ul,tar].  The  boundary  conditions  given  by 
(11.18)  and  (11.19)  require  the  target  to  start  and  end  at  the  given  initial  and  final 
speeds,  vl0  and  vlf.  The  boundary  condition  (11.20)  requires  the  target  to  start  at 
the  starting  position,  yl0(a),  at  the  target’s  starting  time,  tl0(a).  The  final  boundary 
condition  (11.21)  requires  the  target  to  hit  the  HVU  at  the  final  time,  tf. 

We  solve  this  optimal  control  problem  via  a  direct  method  that  fits  seventh- 
order  polynomials  to  the  target  trajectories  in  a  manner  similar  to  that  found  in 
Yakimenko  (2000)  and  Ghabchcloo  et  al.  (2009).  We  note  that  the  optimal  control 
problem  used  to  obtain  the  target  trajectories  is  time-optimal,  and  we  are  concerned 
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with  search  during  the  planning  horizon  [0,1].  This  will  require  truncation  of  the 
target  trajectories  after  one  hour  of  travel  time,  and  is  further  discussed  in  Section 
V.B.l. 

We  are  now  ready  to  state  the  generalized  optimal  control  problems  that  we 
consider  in  this  dissertation. 

D.  GENERALIZED  OPTIMAL  CONTROL  PROBLEMS 

Our  goal  is  to  find  trajectories  for  the  K  searchers  that  optimize  the  expressions 
(II. 6),  (II. 7),  (II. 10),  and  (11.12).  We  assume  that  the  motion  of  the  kth  searcher, 
which  we  refer  to  as  its  “physical”  dynamics,  is  governed  by  the  differential  equation 

xk(t)  =  hk(xk(t),  uk(t)),  t  G  [0, 1],  a:fc(0)  =  £k,  (11.22) 

where  £ k  is  the  vector  of  initial  conditions  for  the  kth  searcher,  which  could  include 
things  such  as  the  initial  position  and  the  initial  heading  of  the  searcher,  the  control 
uk(t)  G  is  the  control  input  to  the  kth  searcher  at  time  t,  which  could  be  the  rate 
of  change  of  the  heading  of  the  searcher,  and  therefore  hk  :  Mn  x  Mmfc  — >  Mn.  We  also 
define  m  =  Yhk= i  £  =  (£\  •••■>  £A)T ,  and  u(t)  =  (ul(t)T , ...,  uK ( t)T)T ,  and  therefore 

the  collection  of  hk  is  given  by  h  :  WlK  x  Mm  — >  MnA.  We  can  also  handle  time- 
varying  systems  and  arbitrary  planning  horizons  by  using  standard  transcriptions; 
see  for  example  p.  493  in  Polak  (1997). 

In  order  to  completely  state  the  generalized  optimal  control  problems,  it  is 
necessary  to  limit  the  searcher  controls  to  certain  spaces  which  we  define  in  Chap¬ 
ter  III.  For  now,  we  give  preliminary  definitions  of  the  generalized  optimal  control 
problems. 

We  define  x’J,fe(i),  t  G  [0,1],  as  the  solution  to  (11.22)  with  r/  =  (£,w)  as  the 
input,  and  let  xv(t)  =  [xv’1(t)T,  ...,x1]’K(t)T)T .  We  assume  that  the  solution  xv’k(t), 
t  G  [0, 1],  exists  and  is  unique  based  on  assumptions  we  will  make  in  Chapter  III.  For 
coordinated  targets,  we  refer  to  the  problem  that  minimizes  the  probability  that  all 
of  the  searchers  fail  to  detect  any  of  the  targets  during  [0, 1]  by  choosing  the  best 
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as  the  general  target  problem  ( GTP ).  From  (II. 6),  we  have  the  following  generalized 
optimal  control  problem  formulation: 


(GTP)  :  min 

v 


exp 


/  ,1  K  L 

-/  ££<-‘V‘ 

\  •j/°  fc=!  ;=i 


xv,k(t),y  (t]  a))dt  (j)(a)da  >  .  (11.23) 


We  note  that  constraints  on  searcher  controls  and  initial  conditions  will  be  given  in 
Chapter  III.  The  constrained  general  target  problem  ( GTPC )  is  exactly  the  same  as 
(GTP),  with  the  addition  of 


u(t)  eU,te  [0,  l], 


(11.24) 


where  U  is  a  convex,  compact  subset  of  Mm.  For  coordinated  targets,  we  refer  to 

the  problem  that  maximizes  the  expected  number  of  targets  detected  during  the  time 

period  [0,1]  by  choosing  the  best  r/  as  the  general  target  problem  (GTPe).  From 

(II. 7),  we  have  the  foilwing  generalized  optimal  control  problem  formulation: 

k 


(GTPe)  :  min  E 


i=i 


exp 


f  y ^rk’1  (xv,k (t) ,  yl (t]  a))dt]  4>(a)da 

k= i  ' 


(11.25) 


The  constrained  general  target  problem  (GT Pc,e)  is  exactly  the  same  as  (GT P)  with 
the  addition  of  (11.24). 

For  independent  targets,  we  refer  to  the  problem  that  minimizes  the  probability 
that  all  of  the  searchers  fail  to  detect  any  of  the  targets  during  [0, 1]  by  choosing  the 
best  r)  as  (ITPP),  and  the  problem  that  maximizes  the  expected  number  of  targets 
detected  during  the  time  period  [0, 1]  by  choosing  the  best  rj  as  the  independent 
target  problem  (ITPe).  From  (II.  10)  and  (11.12),  respectively,  we  get  the  following 
generalized  optimal  control  problem  formulations: 


(ITPP)  :  min  n 


.  i=i 


,1  K 


xl£A 


exp 


rk’1  (xv’k (t) ,  yl (t;  al))dt  4>l(al)dal 


and 

(ITPe)  :  min  IE 


i=i 


k=  1 


-  /  eXP  (  -  /  £ 

Jal&A  \  Jo  k=1 


(11.26) 


rk’l(xr,,k(t),yl(t ;  al))dt  4>l(al)dal 


(I 


,27) 
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The  constrained  independent  target  problems  ( ITPC,P )  and  (PTPc,e)  are  exactly  the 
same  as  ( ITPP )  and  (PTPe),  respectively,  with  the  addition  of  (11.24).  We  again  note 
that  constraints  on  searcher  controls  and  initial  conditions  for  (PTPP)  and  (JTPe) 
will  be  given  in  Chapter  III. 
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III. 


CONSISTENT  APPROXIMATIONS 


The  problems  ( GTP ),  ( GTPC ),  ( GTPe ),  ( GTPc’e ),  ( ITPP ),  ( ITPC'P ),  (JTPe), 
and  ( ITPc,e )  defined  in  Chapter  II  are  infinite  dimensional  in  both  time  and  space. 
In  order  to  numerically  solve  these  problems,  some  form  of  discretization  is  necessary. 
For  the  discretization  scheme  to  be  useful,  it  must  lead  to  implcmentable  algorithms 
that  can  solve  the  discretized  problems  in  a  reasonable  amount  of  time,  and  the  result¬ 
ing  solutions  must  correspond  to  the  solutions  of  the  original  problems  in  some  sense. 
The  discretization  schemes  provided  in  this  chapter  meet  these  requirements.  In  this 
chapter,  we  also  formally  define  the  term  consistent  approximation,  used  to  describe 
the  relationship  between  solutions  obtained  using  finite-dimensional  approximating 
problems  to  the  solutions  of  the  original  infinite-dimensional  problems. 

As  alluded  to  in  Chapter  II,  the  problems  (GTP),  ( GTPC ),  ( GTPe ),  ( GTPc’e ), 
(. ITPP ),  (ITPC,P),  ( ITPe ),  and  (ITPc,e)  are  well  defined  only  if  the  allowable  searcher 
controls  are  restricted  to  certain  spaces.  This  chapter  begins  with  specific  definitions 
of  those  spaces.  The  remainder  of  the  chapter  is  then  split  into  two  main  sections, 
beginning  with  the  treatment  of  coordinated  targets  and  followed  by  independent  tar¬ 
gets.  Both  of  these  main  sections  proceed  in  a  similar  manner,  as  follows.  First,  we 
define  an  “information  state,”  which  we  use  to  write  the  generalized  optimal  control 
problems  in  terms  of  the  spaces  defined  in  Section  III.A.  Next,  we  state  our  assump¬ 
tions  and  define  optimality  conditions  for  the  generalized  optimal  control  problems. 
Then,  we  develop  consistent  approximations  for  the  time-discretized  search  problems. 
Finally,  we  show  that  the  time-  and  space-discretized  search  problems  are  consistent 
approximations  for  the  original,  continuous  time-and-space  search  problems. 

A.  CONTROL  INPUT 

We  define  the  problems  (GTP),  (GTPC),  (ITPP),  (ITPC’P),  (ITPe),  and 
(ITPc,e)  using  the  following  spaces,  which  are  adopted  from  Chapters  4  and  5  of 
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Polak  (1997)  and  described  here  for  the  sake  of  completeness.  For  the  sake  of  brevity, 
we  do  not  explicitly  deal  with  the  problems  ( GTPe )  and  ( GTPc,e )  in  this  chapter. 
We  note,  however,  that  results  developed  in  Section  III. C  for  the  problems  ( ITPe ) 
and  ( ITPc,e )  can  be  trivially  extended  to  include  ( GTPe )  and  ( GTPc'e ),  respectively, 
if  a1  is  replaced  by  a  and  4>\-)  is  replaced  by  4>(-).  We  assume  that  the  control  u  is 
an  element  of  a  subset  of  L™[ 0, 1],  the  space  of  Lebesgue  square-integrable  functions 
from  [0, 1]  into  Mm.  Hence,  initial  condition  and  control  input  pairs  are  elements  of 
the  space 

F2-K"xL“[0,1],  (III.l) 

We  denote  the  L^O,!]  inner  product  for  any  pair  of  functions  u,v  G  L™[ 0,1]  by 
(u,v)2  =  f*  (u(t),v(t))  dt,  with  (•,  •)  denoting  the  Euclidean  inner  product.  The 
corresponding  norms  ||tt||2  =  (u,u)^2  and  ||  •  ||  —  (-,-)1^2.  For  any  77  =  (£,  w)  G  H2, 
with  £  G  Mn  and  u  G  L™[ 0,1],  and  any  rj  =  (£',t/)  G  H2j  with  £'  G  Mn  and 
v!  G  L™[ 0, 1],  we  define  the  inner  product  and  norm  on  H2,  respectively,  by 


(v,v')h2  =  it,?)  +  <«,«')  2 

(111.2) 

IWk  =  (M2  +  Wl2)1/2- 

(III.3) 

We  further  restrict  the  control  u  to  be  an  element  of  L™[0, 1],  the  space  of 
essentially  bounded,  measurable  functions  from  [0, 1]  into  Mm,  but  find  it  convenient 
to  retain  the  inner  product  and  norm  of  L™[ 0, 1].  Hence,  as  in  Section  5.6  of  Polak 
(1997),  we  define  the  pre-Hilbert  space 

H^2^RnxLZ,2[0,l],  (III.4) 

where  L™  2[0, 1]  denotes  the  space  0, 1]  equipped  with  (•,  -)2  and  ||  •  ||2.  We  note 
that  II.y-  2  is  dense  in  H2. 
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We  also  assume  that  there  exists  a  pmax  <  oo  such  that  controls  of  interest  at 
all  times  are  located  in  the  interior  of  £>(0,  pmax)  —  {v  G  Mm  |  ||n||  <  pmax}.  Therefore, 
we  focus  on  the  space 

H  =  x  U  C  #00,2  (III. 5) 

of  initial  conditions  and  control  inputs,  where 

u  4  {u  e  L™  2[o,  i]  I  u(t)  e  #(0,pmax), vt  e  [o,  l]} .  (m.6) 

When  dealing  with  differentiability  statements  we  restrict  ourselves  to  the 

subset 

H°  =  Mn  x  U°  C  H.  (III. 7) 

where 

U°  -  {«£  L^2[ 0, 1]  |  u(t)  G  #(0,7/w),  Vt  G  [0, 1]}  ,  (III.8) 

and  7  G  (0, 1)  is  close  to  one.  It  is  clear  that,  when  pmax  — >  oo,  H°  “fills”  #oo,2- 

Finally,  we  allow  for  the  inclusion  of  a  specific  type  of  pointwise  control  con¬ 
straint  of  the  form  u(t)  G  U  for  all  t  G  [0,1],  where  U  C  Mm  is  a  convex  compact 
subset  of  £>((),  7/>ma.x )  -  This  means  we  only  consider  pointwise  control  constraints  of 
the  form  u  G  Uc,  where 

uc  =  {«  G  U°  I  u(t)  euc  #(0, 7/w),  vt  G  [0, 1]}  .  (III.9) 

We  also  use  the  notation 

Hc  =  P  x  Uc.  (III.  10) 

B.  COORDINATED  TARGETS 

1.  Information  State  and  Optimal  Control  Problems 

In  this  section,  we  define  an  “information  state”  which  we  use  to  write  the 
generalized  optimal  control  problems  in  terms  of  the  spaces  defined  in  Section  III. A. 
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For  a  given  r/  e  H.  recall  that  xv,k(t),  t  £  [0, 1],  is  the  solution  to  (11.22)  with  rj  as  the 
input.  Based  on  (11.23)  we  define  the  objective  function  /  :  H  — y  M  for  any  rj  £  H  by 


„i  K  L 


f(v)  =  exp  -  EE  rk’l(xv’k(t),yl(t;a))dt  I  <p(a)da.  (III. 11) 


k=  1  1=1 


In  order  to  simplify  the  notation  in  (III.  11)  as  well  as  facilitate  the  theoretical 
development  that  follows,  we  find  it  useful  to  define  a  parametric  “information  state” 
denoted  by  z7,(t]a )  £  M,  t  £  [0,1],  a  £  A.  For  any  a  £  A,  t  £  [0,1],  and  set 
of  searcher  trajectories  {xv,k(s),  0  <  s  <  t},  k  =  1,2, . ..,/!,  zv(t]a)  represents  the 
cumulative  detection  rate  given  those  searcher  trajectories  and  vector  of  parameters 


a  and  is  given  by 


rt  K  L 


«;“)=/ EE  rk’1  (xv’k(s),yl(s]a))  ds ,  (III.  12) 

k= i  i=i 


or  equivalently  by  the  differential  equation 


zv(s]a)  =  ^  (xv’k(s),  yl(s;  a))  V  s  £  [0,t], 


(III. 13) 


fc=i  j=i 


with  zv(0;a)  =  0.  Using  this  notation,  for  any  rj  £  H,  (III. 11)  simplifies  to 

f(v)—  [  exp  (— zv(l;  a))  4>(a)da.  (III. 14) 

J  a£A 

It  is  useful  to  simplify  the  notation  in  (III.  14)  even  further.  We  begin  by 
defining  the  notation  £  =  (£1, .  £K,  0)T  £  MnA+1,  where  £  is  a  vector  of  initial  states 
for  the  searchers  such  that  the  first  K  elements  correspond  to  the  initial  “physical” 
states  contained  in  rj  and  the  final  element  corresponds  to  the  initial  “information” 
state.  We  then  define  the  function  F  :  MnA+1  x  MnA"+1  — y  M  such  that  for  any 
|  £  M"a+1  and  x  £  MnA+1,  where  x  =  (x_i,  z)T ,  with  x_i  £  M.nK ,  and  z  £  M, 


F(£,x)±e-Z. 


(III. 15) 


To  complete  our  notational  simplification,  for  any  a  £  A,  we  also  define  the  function 
/(•;«)  :H->M  by 

f(w<x) -F  (i,x’l(t;a))  ,  (III.16) 
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where  xv(t;  a)  is  an  augmented  state  defined  by  combining  the  “physical”  states  with 
the  “information”  state  as  follows 


^(t-a)  =  |  '  |  G  WnK+\ 

zv(t;a) 


(III. 17) 


where  xv(t)  =  (xv’1(t)T, ...,  xv,K  (t)T)T .  Using  this  notation,  for  any  rj  G  H  the  objec¬ 
tive  function  in  (III.  14)  simplifies  to 


/(h)-  /  f(v,a)(/)(a)da. 

Ja&A 


(III. 18) 


We  now  complete  the  definitions  of  ( GTP )  and  ( GTPC ),  which  were  prelimi¬ 
narily  stated  in  Section  II. D.  Using  the  spaces  defined  in  Section  III. A  we  let 


and 


(GTP)  min  /(?/), 

r?eH° 


(< GTPC )  min  f(rj). 

ijeHc 


(III. 19) 


(III. 20) 


2.  Optimality  Conditions 

In  this  section,  we  state  our  assumptions  and  give  optimality  conditions  for 
(GTP)  and  (GTPC).  We  begin  by  deriving  parameterized  differential  equations  of 
the  augmented  dynamics  in  terms  of  the  augmented  state,  x(t ;  a),  defined  in  (III. 17). 
For  t  G  [0, 1]  and  a  G  A,  we  define 

/  o.lf+W  \ 


h(x(t),u(t );  a)  = 


h1  (x1  (t) ,  u1  (t)) 


nnK+l 


hK (xK(t),uK  ( t )) 

\ZLELi  rk*(x\tU(t-a))) 

where  u(t)  =  (ul(t)T, ...,  uK (t)T)J  .  Hence, 


(III. 21) 


x(t]  a)  =  h(x(t),u(t );  a),  t  G  [0, 1],  5/0;  a)  = 


(III. 22) 
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We  note  that  xvf]  •)  is  the  solution  of  (III. 22)  when  the  input  is  y  =  (£,u),  and 
the  augmented  initial  conditions  are  given  by  £  =  (£r,  0)T.  We  next  state  a  series 
of  assumptions,  beginning  with  those  related  to  (/>(■)  and  yl(-;  •).  It  should  be  noted 
that  in  these  assumptions  and  throughout  the  dissertation  components  of  vectors  are 
indicated  by  subscripts  on  variables. 

Assumption  III.l.  We  assume  that  0(-)  is  four  times  continuously  differentiable.  □ 

Assumption  III. 2.  We  assume  that 

(i)  for  all  t  G  [0,1],  yl{t\-)  is  four  times  continuously  differentiable  for  all  l  = 
1,  2, ...,  L,  and 

(ii)  y\-,a)  is  Lebesgue  integrable  on  [0, 1],  for  all  a  G  A. 

□ 

The  compactness  of  A  and  Assumptions  III.l  and  III. 2,  respectively,  ensure  that  the 
partial  derivatives  up  to  fourth-order  of  (/>(•)  and  yl(t]  •),  t  G  [0, 1],  are  bounded.  The 
assumptions  that  ff)  and  yl(t\  •),  t  G  [0, 1],  are  four  times  continuously  differentiable 
and  the  consequence  regarding  the  bounds  on  their  partial  derivatives  up  to  fourth- 
order  are  sufficient  to  complete  the  proofs  of  convergence  in  this  chapter  based  on  the 
later  decision  to  use  Composite  Simpson’s  rule  for  our  numerical  spatial  integration 
scheme;  see  Section  III. B. 3b.  If  we  had  decided  to  use  another  numerical  spatial 
integration  scheme,  such  as  Trapezoidal  rule,  the  assumptions  on  0(-)  and  yl(t\  •),  t  G 
[0, 1],  could  be  relaxed  to  being  twice  continuously  differentiable  with  the  consequence 
that  their  partial  derivatives  up  to  second-order  are  bounded.  The  assumption  that 
yl(-,a)  is  Lebesgue  integrable  on  [0, 1],  for  all  a  G  A  is  sufficient  to  ensure  that  the 
composite  function  rk,l(xk(-),yl(-',a))  is  Lebesgue  integrable  on  [0,1],  for  all  a  £  A, 
under  the  assumption  that  rk,l(-,  •)  is  continuous,  and  mild  assumptions  on  xk( •)  as 
given  in  Assumption  III.3;  see  Azagra  et  ah  (2009). 

The  next  set  of  assumptions  are  related  to  rk,l(-,-)  and  /r(-,-;-).  In  these 
assumptions,  and  throughout  the  dissertation,  we  let  hk(-,-)  denote  the  n  x  n  matrix 
of  partial  derivatives  with  element  i,  j  given  by  ^  ’  ,  and  let  hkf,  •)  denote  the  nxm 


Qljfc  /  \ 

matrix  of  partial  derivatives  with  element  (i,j)  given  by  ^  ’ ; .  We  also  adopt  the 
matrix  norm  ||A||  =  max|uii=1  ||At;||,  for  any  matrix  A  G  Mmxn,  where  v  G  Mn  is  a 
column  vector.  We  note  that  for  matrices  A  G  Mmxn,  B  G  Mnxr,  and  column  vector 
x  G  Mn,  we  have  that  ||Ax||  <  \\A\\ ||a;||,  \\AB\\  <  \\A\\ \\B\\,  and  ||a;afr||  =  ||a;||2  (see  for 
example  p.  26  in  Gill  et  ah,  1991).  We  also  adopt  the  notation 


hx(x(t),u(t)-,a )  = 


VEf=iE"=i  S7xrk\xk{t\y\t-a))T ) 


h*(x(t),u(t)Y 

'L  v-7  ^k,l(rY,kl+\  n,l/ 


(III. 23) 


where  hx(x(t),u(t );  a)  is  a  (riK  +  1)  x  n  matrix  and 

(  hl(x(t),u(t))T\ 


hu(x(t),u(t)- a)  = 


hu(x(t),u(t))T 

\  ° 


{111.24) 


where  hu{x{t),u{t)]  a)  is  a  (nK  +  1)  x  m  matrix. 

Assumption  III. 3.  We  assume  for  all  k  =  1,2, ...,  K  and  l  =  1,  2, ...,  L  that 

(i)  there  exists  a  Cr  <  oo  such  that  for  all  xk  G  Mn  and  yl  G  Mny 


0  <rk\xk,yl)<Cr, 


(III. 25) 


(ii)  rk,l{-,yl)  is  continuously  differentiable  for  all  yl  G  Mnj/, 

(iii)  rk,l{xk,  •)  is  four  times  continuously  differentiable  for  all  xk  G  Mn, 

(iv)  Vxrk,l{xk,  •)  is  four  times  continuously  differentiable  for  all  xk  G  M", 

(v)  hk{-,  •)  is  continuously  differentiable 

(vi)  there  exist  Cr i  <  oo,  Cr 2  <  oo,  and  Cr3  <  oo  such  that  for  all  xk  G  Mn  and 
yl  G  Mny, 

<  Crl  V*  =  1,2, j  =  1,2, 3, 4,  (III.26) 

<  Cr2  Vi  =  1,2, j  =  l,2,3,4,  (III.27) 


dirk’l(xk,  yl ) 
dy\ 

di+1rk’1  {xk ,  yl) 
dxdy{ 
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and 


\\Vxrk’l(x\yl)\\  <Cr3, 

(III. 28) 

there  exists  a  constant  K  G  [l,oo)  such  that  for  all  x' , 
B(0,  pmax),  and  a  G  A,  the  following  hold: 

x"  G  Mn,  v\  v"  G 

J77 

1 

IA 

7T 

1 

sy 

+ 

AT 

1 

(III. 29) 

|| hx{x' ,v'\ot)  -  hx(x",v";a)\\  <  K  [||x'  -  x"||  +  \\v'  -  v"\\] , 

and 

(III. 30) 

|| hu(x',v';a)  -  hu(x",  v"\ a)  <  K  [||x'  -  x"||  +  | \v'  -  v"\  ] . 

(III. 31) 

□ 

We  note  that  (III. 29)  implies  that  there  exists  a  K'  <  oo  such  that  for  all  x'  G  Mn, 
and  v  G  5(0,pmax), 

\\h(x',v)\\  <  K'  [||a/||  +  1] .  (III. 32) 

The  assumptions  about  the  detection  rate  function,  rk,l(-,-),  in  Assumption 
III. 3  are  not  overly  restrictive  as  they  allow  for  the  use  of  many  types  of  sensor 
models,  and  are  similar  to  those  used  by  other  researchers  (see  Chung  et  al.,  2010, 
for  example).  Assumptions  III. 3  (v)  and  (vii)  are  standard  assumptions  that  parallel 
those  adopted  in  Assumption  5.6.2  of  Polak  (1997).  Assumption  III. 3  (vii)  guarantees 
a  unique  solution  to  the  differential  equations  governing  the  searcher  dynamics  given 
by  (III. 22). 

We  next  show  that  /(•)  is  Gateaux  differentiable  on  H°,  but  in  order  to  do 
this  we  need  the  following  intermediate  result. 

Lemma  III. 4.  Suppose  that  Assumptions  III. 2  and  III. 3  are  satisfied,  then  for  any 
a  G  A,  p  G  H°,  and  8rj  G  h?oo,2: 

(a)  /(•;  a)  has  a  Gateaux  differential  Df(r /;  a;  Srj)  at  r)  and  hence  a  directional  deriva¬ 
tive  <7/(77;  a;  Srj),  with  df(rj]  a;  Sp)  =  Df(rj ;  a;  Srj),  given  by 

Df(jr,a\5rj)  =  (v  J {ry,  a) ,  8r])  ,  (HI-33) 

'  /  H2 
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where  the  gradient  Vvf(p]  a)  =  (V^/^;  a),  Wuf(p;  a))T  G  #00,2  is  defined  by 

V^/faa)  =  p"(0;a),  (HI.34) 

Vu/ (77;  a)(£)  =  hu(x^(t),u(t)]cx)Tp^(t]Qi),ior  t  G  [0,1],  (III.35) 


and  ^(t;  a)  is  the  solution  of  the  adjoint  equation 

pv(t-,a)  =  -hx(xv(t),u(t)-,a)Tpv(t-a),  t  G  [0,1], 

(  0  \ 


p,?(l;a)  = 


0 


G  RnK+\ 


(III. 36) 


\“  exp  (—zv(T,  a)) J 


(b)  the  gradient  V ,,/(•;  a)  is  Lipschitz  H-continuous  on  bounded  subsets  of  H°  for 
all  a  G  A  with  Lipschitz  constant 

0 


K 


exp  ( V2KeK  )+K  ( V2Ke*  +  1 


K 


(III. 37) 


where  K  G  [1,  cxd)  is  as  in  Assumption  III.3(vii)  and  K'  <  00  is  as  in  (III. 32). 

(c)  /(•;«)  is  Lipschitz  H-continuous  on  bounded  subsets  of  H°  for  all  a  G  A  with 
Lipschitz  constant  y/2 KeK,  where  K  G  [1,  00)  is  as  in  Assumption  III.3(vii). 


(d)  /(•;  a)  has  a  Frechet  differential  at  77  relative  to  H  that  is  equal  to  Df(r}-,  a;  8rj). 


Proof.  Parts  (a)  and  (d)  result  from  an  application  of  Corollary  5.6.9  in  Polak  (1997). 
Part  (b)  results  from  an  application  of  Corollary  5.6.9  and  the  proof  of  Lemma  5.6.7 
in  Polak  (1997).  For  part  (c),  we  deduce  from  Lemma  5.6.7  in  Polak  (1997)  that 
zn(l\  a)  is  Lipschitz  H-continuous  as  a  function  of  p  with  Lipschitz  constant  \/2Keh . 
Since  zv(  1;  a)  >  0  for  all  77  G  H"  and  a  G  A,  it  follows  that  /(•;« )  is  Lipschitz 
H-continuous  with  Lipschitz  constant  \/2KeK  because  the  magnitude  of  the  slope  of 
the  exponential  function  with  an  argument  in  the  domain  (— 00,  0]  is  bounded  by  one. 
□ 

Proposition  III. 5.  Suppose  that  Assumptions  III.l,  III. 2  and  III. 3  are  satisfied. 
Then,  for  any  77  G  H"  and  Sp  G  L7oo,2,  /(•)  has  a  Gateaux  differential  Df( 77;  5p)  at  p 
given  by 

=  (IIL38) 

where  the  gradient  V  f(p)  is  given  by 

Vf(p)(t)  =  I  Vj(p-,a)(t)<f>(a)da,Vt  G  [0,1].  (IIL39) 

J a&A 
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Proof.  Let  Sr]  E  V  G  H°,  and  a  E  Abe  arbitrary.  There  exists  a  A*  >  0  such 

that  r/  +  XSr]  E  H  for  all  A  E  [0,  A*].  For  A  E  [0,  A*],  consider  the  ratio 


R(a;  r),  Sr],  A)  = 


f{fi  +  XSri;a)  -  ffa  a) 
A 


where  from  Lemma  111.4(a)  we  know  that 


lim  R(a;  rj,  Sr],  A)  =  Df(r /;  a;  5rj)  =  (vf{ry,  a),  5r^j ^  . 


From  Lemma  111.4(c)  we  know  that 


(III. 40) 


(III. 41) 


\R(a]V,5r],\)\  <  L\\5V\\H2,  (III.42) 

with  L  =  a/2 Keh.  We  also  know  that 

V2Kek  [  (fi(a)da  =  V2Kek.  (III.43) 

J  olGA 

Then  because  Srj  is  a  constant  with  respect  to  a,  it  is  clear  from  (III. 42)  and  (III.43) 
that  for  all  A,  R(-;  rj,  Sr],  A)  is  dominated  by  an  integrable  function.  Then  the  Lebesgue 
Dominated  Convergence  Theorem  yields  that 


lim  /  R(or,rj,8r),  \)(j>(a)da  —  /  lim  R(a;  rj,  Sr],  X)cj)(a)da. 


-T0  J aGA 


ga 


We  then  use  (III. 44)  and  (III. 41)  to  obtain  that 

Df(rrM  =  i  + 

A4,o  A 


=  lim 

AJ.0 


*gA 


f(v  +  a)  -  Km  a ) 

A 


4>(a)da 


=  lim  /  R(a;  r],  Sr],  A )4>{a)da 


'''l0  J  aeA 

/  ( V^/ (77;  a),  Sr)\  (j>{a)dot. 

J aeA  N  '  h2 


(III. 44) 


(III. 45) 


Finally,  (  Vr?/(r7;  a),  Sr] \  is  linear  in  Sr]  because  by  Lemma  111.4(d)  f(r]\  a)  is  Frechet 


h2 


differentiable  relative  to  H.  Thus,  (III. 39)  follows  directly  from  (III. 45). 
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Proposition  III. 5  also  holds  under  weaker  assumptions  on  </>(•),  but  because  we 
need  Assumption  III.  1  later  we  adopt  it  here  as  well.  This  issue  regarding  Assumption 
III .  1  also  applies  elsewhere  in  this  chapter.  Our  next  task  is  to  show  that  V/(-)  is 
Lipschitz  H-continuous  on  bounded  subsets  of  H°. 

Lemma  III. 6.  Suppose  that  Assumptions  III.l,  III. 2  and  III. 3  are  satisfied,  then  the 
gradient  V/(-)  is  Lipschitz  H-continuous  on  bounded  subsets  of  H°. 


Proof.  For  any  p'  ,rf'  £  H°,  and  t  £  [0, 1], 


\\S7fW)(t)  ~  Vf{rf'){t)\\=  /  Vvf(p'-a)(t)(j)(a)da-  Vvf(p";a)(t)<l>(a)da 

J aeA  J aeA 

From  Lemma  111.4(b)  we  know  for  any  a  E  A,  V  ,,/(•;  a)  is  Lipschitz  H-continuous  on 


bounded  subsets  of  Hn  with  Lipschitz  constant  eK 
Then, 


'  exp  ( 'V2Ke R^j  +  K  (^2 Kek  +  1 


V^/(V;  cx)(t)(f)(a)da  —  /  Vr?/(ry//;  a)(t)(j){a)da 


exp  (V2Ke 


Because 


vim)  = 

we  know  from  (III. 46)  that 


v£/(u) 

v,/MM 


l)]  \W  -  v"\\h2- 

(III. 46) 

,  t  £  [0,  1], 

(III. 47) 

IIW/W)  -  V5/(,")||2  +  ||V„/(,')(t)  -  V„/(r,'')(()||2 

<  ek‘  exp  ( s/2Kek)  +  K  (V2AVf  +  l)  \\tf  -  t/'||2„, .  (III.48) 

Hence, 

I|V/(t7/)  -  V/(r7,,)||^2 

=  ii  ve/(77/)  -  v?/(r7/,)n2  +  ['wvjwm-vjwmfdt 

Jo 

<  eR'  exp  (^/2KeR^j  +  K  (^/2Keh  +  1  j  jj| p'  —  p”\\2H., 

+  J  eR'  exp  (\/2KeR^j  K  (V2KeR  +  lj  \\p  —  p" \\2Hn  dt 

<  2  eR’  exp  (V2Kek^J  +  K  (V2KeR  +  l)  2  \\rf  -  p"\\2H2  •  (HI-49) 
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Because  K  and  K'  are  both  non-negative,  (III. 49)  implies 

II V/(r/)  -  V/(r/') \\h2  <  V2e«'  [exp  (x/2 Kek)  +  K  [flKek  +  l)]  \\rf  -  r/'||^  , 

(III. 50) 

which  completes  the  proof.  □ 

We  adopt  the  approach  of  Polak  (1997)  (see  section  4.2),  and  state  our  optimal¬ 
ity  conditions  in  terms  of  zeros  of  optimality  functions;  see  also  Section  III. B. 3a.  We 
specifically  define  the  nonpositive  optimality  functions  6  :  H°  — y  M  and  6C  :  H,  — »  M 
as 

m  =  -)iiv/wiik.  (in.61) 

and 

°c(v)  =  min  (V/fa),  rf  -  rj)H2  +  )-  \W  ~  v\\h2  ,  (III.52) 

77  Eric  Zj 

which  define  optimality  conditions  for  ( GTP )  and  ( GTPC ),  respectively. 
Proposition  III. 7.  Suppose  that  Assumptions  III.l,  III. 2  and  III. 3  are  satisfied. 

(a)  #(•)  and  6cfi)  are  H°  -continuous  functions. 

(b)  If  fj  G  H°  is  a  local  minimizer  of  (GTP),  then  6(f))  =  0. 

(c)  If  fj  E  Hc  is  a  local  minimizer  of  ( GTPC ),  then  9c(r 7)  =  0. 

Proof.  The  proof  follows  the  same  arguments  as  those  for  the  proof  of  Theorem  4.2.3 
in  Polak  (1997),  with  Proposition  III. 5  taking  the  place  of  Corollary  5.6.9  from  Polak 
(1997)  and  Lemma  III. 6  taking  the  place  of  Theorem  4.1.3  from  Polak  (1997).  □ 

3.  Consistent  Approximations 

As  discussed  at  the  outset  of  this  chapter,  in  order  to  solve  (GTP)  and  (GTPC), 
some  form  of  discretization  is  necessary.  In  this  section  we  define  the  approximat¬ 
ing  problems  ( GTP V),  (GTPfi),  (GTPNMjNfi,  and  (GTP^M^).  We  also  present 
conditions  that  must  be  satisfied  to  ensure  that  global  minimizers,  local  minimizers, 
and  stationary  points  of  the  approximating  problems  converge  to  global  minimizers, 
local  minimizers,  and  stationary  points,  respectively,  of  the  original  problems.  We 
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divide  our  development  into  two  subsections.  Both  subsections  develop  consistent  ap¬ 
proximations  for  the  pairs  (( GTP),9 )  and  (( GTPC),9C ),  but  the  first  subsection  is  a 
“stepping  stone”  that  only  deals  with  time  discretization  while  the  second  subsection 
considers  time  and  space  discretization.  In  both  subsections,  we  adopt  the  notation 
from  sections  4.3  and  5.6  of  Polak  (1997). 


a.  Time- Discretized  Problems 

Let  N  denote  the  positive  integers,  and  let  A f  be  an  ordered  infinite 
subset  of  N  defined  by 


A' 4  {2'IJ^.  (III. 53) 

To  begin  our  development,  we  must  first  define  an  infinite  set  of  finite-dimensional 
subspaces  HN  C  whose  union  is  dense  in  H^-  For  IV  e  Af  and  j  =  0, 1, ..,  IV  — 

1,  let  the  functions  hn.j  '■  [0, 1]  — »  M,  be  defined  by 

'  VNWte[j/N,  (j  +  l)/N),  if  j<N-2, 
nNj(t)  =  VN  Vt  G  \j/N,  (j  +  1)/IV],  if  j  =  N-  1,  (III. 54) 

0,  otherwise. 

We  also  define  the  subspace  ln  C  L™[0,1],  by 


ln  a  tt€Lyo,i] 


N- 1 


«(■)  =  ^UjirNJ(-),Uj  e  Rm,Vj  =  0, 1, IV  -  1  }  , 
j=o 


(III. 55) 


and  let 


Hn  A  m"  x  Ln  C  7/00,2-  (III.56) 

The  functions  7 tnj(-)  form  a  basis  for  L ^  and  are  dehned  such  that  the  relation  be¬ 
tween  Hn  and  the  Euclidean  space  of  coefficients  used  for  numerical  computation 
is  isometric.  We  use  the  function  space  H n  for  proofs  of  consistency  of  approxima¬ 
tion,  and  the  Euclidean  space  of  coefficients  to  develop  implementable  algorithms. 
The  definition  of  the  Euclidean  space  of  coefficients,  HN,  and  further  discussion  of 
implementable  algorithms  are  provided  in  Chapter  V.  We  also  define  the  sets 

Hjv  =  H  n  Hn,  (III. 57) 
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(III. 58) 


—  H°  n  Hn 

and 

ncN  =  hc  n  hn,  (hi. 59) 

with  H.  H°  and  Hc  defined  in  (III. 5),  (III. 7),  and  (III. 10),  respectively.  Based  on  the 
definition  of  A f ,  it  is  clear  that  the  subspaces  HN  possess  a  desirable  nested  structure. 
This  means  that  for  any  given  N,  N'  G  Af  such  that  N'  >  N,  Hn  C  Hn>- 

We  will  make  use  of  Proposition  4.3.1  from  Polak  (1997),  which  is 
included  here  without  proof  for  the  sake  of  completeness.  Below,  we  use  the  notation 
— 4  to  indicate  convergence  of  a  subsequence  defined  by  Af . 

Proposition  III. 8.  (a)  Let  H°z  denote  the  closure  of  the  set  H".  Then,  H”,  —4  Hj), 
as  N  — >■  oo,  and  (b)  Hrjv  — 4  Hc,  as  N  — >  oo,  where  set  convergence  is  in  the  sense 
of  Painleve-Kuratowski2 ;  as  N  — >  oo  along  the  subsequence  defined  by  Af.  □ 


We  now  consider  the  approximate  solution  of  (11.22)  by  means  of  for¬ 
ward  Euler’s  method.  For  any  rj  =  (f,u)  G  Hn  and  N  G  Af,  we  set  x^fc(0)  =  fk  and 
for  any  j  =  0, 1, ...,  N  —  1,  and  for  all  k  =  1,  2, ...,  K 

xt  (0  +  1  )/N)  -  xtf  (j/N)  =  Ift*  (xtfti/N),  u\j/N))  .  (III. 60) 

Simultaneously,  we  approximately  solve  (III.  13)  also  by  forward  Euler’s 
method.  For  any  g  =  (f,u)  G  HN ,  a  G  A,  and  N  G  Af,  we  set  zvN{ 0;  a)  =  0,  and  for 
any  j  =  0,1,  ...,N  -  1, 

4  ((i  +  1)/N', a )  -  4  WN\ a)  =  xf(j/N),yl(j/N]a)y  (III. 61) 

v  k= i  i=i 


Using  the  discretized  “information  state”  given  by  the  recursion  (III. 61), 
we  define  the  approximate  objective  functions  /at  :  Hat  — >  M  for  any  77  G  Hat  and 
N  E  Af  by 

4('7)-  [  exp  (—2^(1;  °0)  <fi(cx)da.  (III. 62) 

J  aeA 


2See  Definition  5.3.6  in  Polak  (1997). 
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Again,  for  the  sake  of  notational  simplification,  for  any  a  G  A,  we  also  define  the 
functions  fiv(-',  a)  :  H”v  — »  R.  by 

M r  a)  =F  (i,  xvN(t;  a ))  ,  (HI-63) 

where  F(-,  •)  is  as  defined  in  (III. 15)  and  xvN(j/N]a )  is  an  augmented  state  defined 
by 

^(j/iV;  o)  4  |  ^O/'V)  |  £  R»Jr+i  •  =  0j !  JV  -  1,  (III.64) 

where  xvN(j/N)  =  (xj}1  (j / N)T , ...,  xv^K  (j/N)T^j  ,  j  =  0, 1, ...,  N  —  1.  Hence,  for  any 
N  G  A/",  we  define  the  following  approximating  problems 

(< GTPn )  min  /at  (r/),  (III.65) 

and 

(GTP^)  min  ^(77).  (III.66) 

J7eHc,jv 

We  note  that  the  problems  (GTPn)  and  ( GTP ^)  still  have  spatial  integrals  that  have 
not  been  discretized. 

We  want  to  show  that  /jv(-),  e  A/",  are  Gateaux  differentiable  and 
that  their  gradients  are  Lipschitz  continuous.  In  order  to  do  this,  we  begin  with  an 
intermediate  result  about  the  sums  and  products  of  Lipschitz  continuous  functions. 
We  then  prove  an  additional  intermediate  result  which  shows  that  the  Av(-;  •)  are 
Gateaux  differentiable,  and  that  their  gradients  are  Lipschitz  continuous. 

Lemma  III. 9.  Suppose  S  is  a  bounded  subset  of  Hn.  If  F,G  :  S  — >■  H°  are  Lipschitz 
H-continuous  functions  on  S,  then  F  +  G,  cF  for  any  c  €  M,  and  FG  are  also 
Lipschitz  H -continuous  functions  on  S. 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Theorem  4.6.3(b)  in 
Sohrab  (2003).  In  Theorem  4.6.3(b),  however,  the  functions  F,G  are  defined  on  a 
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bounded  interval  of  M  and  our  functions  F,  G  are  defined  on  a  bounded  subset  of 
Hu.  Let  Lf  and  Lq  be  Lipschitz  constants  for  F  and  G,  respectively,  then  for  any 
V,  rf  e  S , 


\\(F  +  G)(V)  -  (F  +  G)(t/)\\H2  <  \\F(v)-F(ff)\\Ha  +  \\G(V)~G(rf)\\Ha 

<  (Lr  +  LG)\\ri-ri'\\H2,  (III.67) 

and 

II (°F)(v)  -  (cF)(rf) \\h2  =  \c\\\F(V)  -  F(r/)lk  <  \c\LF\\r}  -  r/lk-  (III.68) 

Since  S  is  a  bounded  subset  and  F  and  G  are  both  Lipschitz  H-continuous  on  S, 
there  exists  M  >  0  such  that  \\F(r])\\H2  <  M  and  \\G(r])\\H2  <  M  for  all  r/  G  S.  Then 
for  any  77,  7/  G  S  we  have 

\\(FG)(V)  -  (FG)(V')\\H2  <  \\G{V)\\Ha\\F(v)-F(rf)\\Ha 

+  \\fW)\\h2\\g(v)-gW)\\H2 
<  ( MLf  +  MLG)\\n  -  rf\\ h2.  (III. 69) 

□ 

Lemma  III.  10.  Suppose  that  Assumptions  III. 2  and  III. 3  are  satisfied,  and  that 
N  G  A f,  then  for  any  a  G  A, 

(a)  the  function  /jv(-;a)  :  — »  M,  is  Gateaux  differentiable,  and  the  gradient 

V/v(h;  a)  =  VufN(r]-,a)S)  G  HN  is  given  by 

V?/v(G«)  =  pl(0’,a),  (III. 70) 

N—l 

VufN(rp,a)(t)  =  ^2^NU/N-,a)nNJ(t),  t  G  [0, 1],  (HI-71) 

3= 0 

where 

TtvOV-W;  «)  =  ^^~hu(xvN(j/N),u(j/N);a)TpnN((j  +  1)/N;a), 

j  —  0, 1,  •••,  N—l,  (III. 72) 
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and  pvN(-',  a)  is  determined  by  the  adjoint  equation 


pnU/n ; «)  -  Pjy(t7  +  1)/N> «) 

=  ^hx(xvN{j/N),u(J/N)\a)TpvN({j  +  1  )/N;a), 
j  =  0,1,..,  TV-l, 

/  0 

=  q 

\—  exp  a)) 

(b)  For  any  bounded  subset  S  of  H.  there  exists  a  Lipschitz  constant  Ls  <  oo  such 
that,  for  any  N  >  1,  j  —  0, 1,  ...,1V,  77, 7/  G  S  D  H jv,  and  a  G  A, 

In{ 77; «)  -  M7; «)  <  -  Vita*  (111.74) 

and 

V/jv (77;  a)  -  V/tv(7;  «)  <  F5||^  -  7/lk2-  (III. 75) 

H2 

Proof.  The  proof  of  part  (a)  is  the  same  as  the  proof  of  Theorem  5.6.20  in  Polak 
(1997),  so  it  is  not  repeated  here.  For  the  proof  of  part  (b),  we  begin  by  proving 
(III. 74).  From  Theorem  5.6.16  in  Polak  (1997)  we  deduce  that  ^(l;o;)  is  Lipschitz 
continuous  as  a  function  of  rj.  Since  znN{  1;  a)  >  0  for  all  ?/  e  H°N  and  a  G  A,  /jv(-;  a) 
is  Lipschitz  H^-continuous  because  the  magnitude  of  the  slope  of  the  exponential 
function  with  an  argument  in  the  domain  (— oo,  0]  is  bounded  by  one. 

To  show  that  V/jv( S  «)  is  Lipschitz  H ^-continuous  on  bounded  subsets, 
we  look  at  its  component  parts  beginning  with  (III. 70).  To  show  that  V^/at(77;q;)  is 
Lipschitz  Hjv-continuous  on  bounded  subsets  we  proceed  with  an  induction  argument. 
From  (III. 70)  we  see  that  a)  is  equal  to  the  value  of  the  adjoint  at  time  zero, 

pvN( 0;  a).  In  order  to  find  pvN(0 ;  a),  we  use  the  recursion  found  in  (III. 73).  We  know 
that  pnN(  1;  a)  is  Lipschitz  H ^-continuous  as  a  function  of  77  for  the  same  reasons  that 
/jv(-;aO  is  Lipschitz  H ^-continuous  as  a  function  of  77.  From  Assumption  III.3(vii) 
and  Theorem  5.6.16  in  Polak  (1997)  we  also  know  that  hx  ( xvN  (t)  ,u(t)  ;  a)  is  Lipschitz 
H 7v-continuous  as  a  function  of  77  for  any  t  G  [0,1],  and  that  its  Lipschitz  constant, 
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<  oo,  is  independent  of  t  and  N .  Suppose  that  pnN((j  +  1  )/N;  a)  is  Lipschitz  Hat- 
continuous  as  a  function  of  r)  and  that  its  Lipschitz  constant,  Lp  <  oo,  is  independent 
of  N.  Because  hx  {xvN  (t) ,  u  (t) ;  a)  and  pvN((j  +  1  )/N;a)  are  both  Lipschitz  Hat- 
continuous  for  all  r/  e  S,  and  S'  is  a  bounded  subset  of  H,  there  exists  M  <  oo 
such  that  || hx  (xvN  (t) ,  u  (t) ;  a)  ||  <  M  and  \\pvN((j  +  1  )/N;  a)||  <  M.  Then,  based  on 
(III. 73)  and  Lemma  III.9,  we  have  that 


pnU/n ; «)  -  pnU/n ; «) 


< 

+ 

< 


Pn(U  +  !)/W;  a)  -  p%((j  +  1)/N;  a) 

^  hx(x^(j/N),u(j/N);  a)TpvN((j  +  1  )/iV;  a) 
hx{x%(j /N),u(j /N)-a)Tp%((j  +  1  )/N;  a) 

Lp\\v  ~  v'\\h2  +  +  MLP) \\rj  -  r/|| h2 

(  m(^  +  i)\ 

Lp  1  +  — H - L  111//-  rfW  (III-76) 


By  doing  another  step  in  the  backward  recursion,  we  hnd  that 


M  (  ^  +  1 

<  LP  |  1  + - h - -  )  \\v  -  v'\\ H2 


<  Lp  I  1  + 


N 


M  (  ^  +  1 

L/p 


N 


N 


\\v-v'\ 


h2- 


(III.  77) 


Let  Km  =  M  +  lj.  There  then  exists  an  N  e  Id  such  that  for  all  N  >  N, 
(l  +  ^l)N  <  2eKM .  Then,  for  any  N  >  N,  we  have 

pKU  - 1)/^; «)  -  p%(U  ~  i)/^; «)  <  2  -  Vlk-  (hi. 78) 


For  values  of  iV  smaller  than  N,  we  define 

N-i 


Si  =  1  + 


K 


M 


N-i 


,  i  =  1,2,  ...,iV  -  1, 


(III. 79) 
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and 


Ks  =  max{2,  max  5*}.  (III. 80) 

i=l,2,...,N-l 

Then, 


<  K5LpeKM\\r)  -  r)'\\H2. 


(III. 81) 


Hence,  it  follows  by  induction  that  V^/at(-;q;)  is  Lipschitz  HAr-continuous,  and  that 
the  Lipschitz  constant  is  independent  of  N. 

Next,  we  consider  V.u,/at(-;  a)  given  in  (III. 71).  From  (III. 54)  we  know 
that  7Ta Tj(t)  is  either  0  or  y/N .  The  ^  factor  in  the  definition  of  r)vN{-)  in  (III. 72) 
combined  with  the  n^^it)  value  of  0  or  yfN  ensures  that  Vu/Ar(-;a)  is  independent 
of  N .  From  Assumption  III.3(vii)  and  Theorem  5.6.16  in  Polak  (1997)  we  find  that 
hu  {xvN  (t) ,  u  (t) ;  a)  is  Lipschitz  H-continuous  as  a  function  of  rj  for  any  t  G  [0,1], 
and  that  the  Lipschitz  constant  is  independent  of  t  and  N.  Based  on  the  induction 
argument  given  in  (III. 76)  through  (III. 81),  we  know  that  p1’1N(ij^~]  a)  is  Lipschitz 
LLv-continuous  as  a  function  of  r)  for  all  j  =  0, 1,  ...,N  —  2,  and  that  the  Lipschitz 
constant  is  independent  of  N.  From  (III. 54)  we  see  that  the  summation  in  (III. 71) 
is  zero  for  all  terms  except  where  t  is  between  j/N  and  (j  +  1)/AL  For  these  non¬ 
zero  terms,  Vb/jv(-;  a)  is  composed  of  products  of  Lipschitz  HAr-continuous  functions 
whose  Lipschitz  constants  are  independent  of  N,  so  by  Lemma  III. 9,  Vu/at(^;q;)  is 
Lipschitz  H w-cont.inuous,  and  its  Lipschitz  constant  is  independent  of  N.  This  means 
that  both  components  of  V/at^;  a)  are  Lipschitz  HAr-continuous  and  their  Lipschtiz 
constants  are  independent  of  N  and  a,  so  the  proof  is  complete.  □ 

Proposition  III. 11.  Suppose  that  Assumptions  III.l,  III. 2  and  III. 3  are  satisfied, 
and  N  e  Af,  then  for  any  rj  G  H^r  and  5r /  G  L7oo,27  /v(-)  has  a  Gateaux  differential 

DfN (r}\Sr})  =  (V fN(v)i  ^v)h2>  where 

V fN(rj)(t)  =  [  VvfN(r}',<x)(t)<l>{(x)da,Vt  G  [0, 1].  (III. 82) 

JaeA 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Proposition  III. 5  with 
/(•;•)  replaced  by  /at(*;  •),  so  it  is  not  repeated  here.  □ 
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Next,  we  show  that  V/jv(-)  is  Lipschitz  H^r-continuous  on  bounded 
subsets  of  H'y- 

Lemma  III. 12.  Suppose  that  Assumptions  III.l,  III. 2  and  III. 3  are  satisfied,  then 
the  gradient  V/jv(-)  is  Lipschitz  HN- continuous  on  bounded  subsets  ofH.%. 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Lemma  III. 6  with  /(•;  •) 
replaced  by  /' v (•;■),  so  it  is  not  repeated  here.  □ 

As  in  Section  III.B.2,  we  state  our  optimality  conditions  in  terms  of 
zeros  of  optimality  functions.  For  any  N  G  A f,  we  define  nonpositive  optimality 
functions  9 n  ■  — y  M  and  0CN  :  HC)JV  — >  M  by 

Ml)  =  -i||VM,)||2ffa,  (III. 83) 

and 

0n(v)  =  niin  (V/^??),  ?/  -  v)h2  +  \  II v'  ~  v\\h2  ,  (III.84) 

T]  GxT.c,1V  ^ 

which  characterizes  stationary  points  of  (GTPn)  and  (GTPrN),  respectively. 
Proposition  III. 13.  Suppose  that  Assumptions  III.l,  III. 2  and  III. 3  are  satisfied. 

(a)  9]\r(-)  and  9CN(-)  are  H'y-continuous  functions. 

(b)  If  fj  G  is  a  local  minimizer  of  ( GTPN ),  then  ^(f/)  =  0. 

(c)  If  r )  G  HCjjv  is  a  local  minimizer  of  (GTPfj),  then  0%{fj)  =  0. 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Proposition  1.1.6  in 
Polak  (1997),  with  the  norms  and  inner  products  replaced  with  their  H-2  equivalents. 

□ 

We  are  now  ready  to  develop  proofs  for  consistency  of  approximation  for 
the  pairs  (( GTPn),9n )  in  the  sequence  {((GTPjsr),  9n)}ngjv-  Because  we  deal  with 
more  than  one  type  of  problem  and  its  corresponding  approximation,  it  is  simpler  to 
define  consistent  approximations  and  epi-convergence  using  abstract  problems.  We 
adopt  the  notation  of  Section  3.3.1  in  Polak  (1997)  and  let  B  be  a  norrned  linear 
space,  with  norm  ||  •  ||^,  and  let  M  =  M  U  {— oo,  +oo}.  We  then  define  the  problem 

(P)  min  f(x),  (III. 85) 
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where  /  :  B  — >  M  is  lower  semicontinuous  and  X  C  B  is  a  constraint  set.  Then  let 
{Bn}n&m  be  a  family  of  finite-dimensional  subspaces  of  B  such  that  Bn,  C  £>y2,  for 
all  Ni  <  N2  G  AT,  and  Uyey,By  is  dense  in  B.  For  all  N  G  N,  let  /y  :  Bn  — >  M  be 
a  lower  semicontinuous  function  that  approximates  /(•)  on  Bn,  and  let  XN  c  Bn  be 
an  approximation  to  A".  We  then  define  the  family  of  approximating  problems 

(Py)  min  fN(x),  N  E  J\f.  (111.86) 

xG.Yjv 

Finally,  we  let  Xci  denote  the  closure  of  X  and  we  define  the  problem  (Pci)  by 

(Pd)  min  f(x),  (111.87) 

xGXci 

which  may  not  be  epi-graphically  equivalent  to  the  problem  (P),  but  which  we  will 
assume  is  equivalent  to  (P)  in  the  sense  that  it  has  the  same  local  and  global  minimiz- 
ers.  Before  we  define  consistent  approximations,  we  must  first  define  what  it  means 
for  a  function  to  be  an  optimality  function.  As  on  page  398  of  Polak  (1997),  we  define 
an  optimality  function  as  follows: 

Definition  III.l.  Let  5  be  a  subset  of  B  such  that  A"  C  S.  We  will  say  that  a 
function  6  :  S  — >  M  is  an  optimality  function  for  (P)  if 

(i)  O(-)  is  sequentially  upper  semi-continuous, 

(ii)  9(x)  <0  for  all  x  G  S,  and 

(hi)  if  a:  is  a  local  minimize!'  of  (P),  then  0(x)  =  0. 

Similarly,  for  all  N  G  Af,  let  Sn  be  a  subset  of  Bn  D  S  such  that  Xn  C  Sn  C  S.  We 
will  say  that  a  function  9n  '■  Sn  — >  K  is  an  optimality  function  for  (Py)  if 

(i)  0N(-)  is  sequentially  upper  semi-continuous, 

(ii)  0n(x)  <  0  for  all  x  G  Sn,  and 

(iii)  if  x n  is  a  local  minimizer  of  (Py),  then  dy(£y)  =  0.  □ 

Then,  as  on  page  399  of  Polak  (1997),  we  define  consistent  approximations  as  follows: 
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Definition  III. 2.  Consider  the  problems  (P),  (P/v),  and  (Pd)  defined  in  (III. 85), 
(III. 86),  and  (III. 87),  respectively.  Let  S'  be  a  subset  of  B  such  that  X  C  S  and,  in 
addition,  Xn  C  S  for  all  N  G  A f .  Next,  for  all  N  G  A f,  let  Sn  be  a  subset  of  S'  f)  Bn 
such  that  Xn  C  Sn  C  S.  Finally,  let  9a  :  S  — »  M  and  :  5jv  — >  M,  A"  G  A/”,  be 
optimality  functions  for  (P)  and  (P/v),  respectively.  We  say  that  the  pairs  ((P/v),  0^), 
in  the  sequence  {(Pn , @n)} NeAT  are  consistent  approximations  for  the  pair  (( P),9a )  if 

(i)  either  (P/v)  epi-converges  to  (P)  or  (P/v)  epi-converges  to  (Pcz),  as  N  — »  oo,  and 

(ii)  for  any  infinite  sequence  {2vv}jVeA:,  A”  <7  A/”,  with  a: at  G  Sat  for  all  N  G  A",  such 

that  xn  — >■  £  as  N  — >  oo,  limsup^^.^  9%(xn)  <  9a(x).  □ 

We  base  our  proofs  of  epi-convergence  on  satisfying  the  conditions  of 
Theorem  3.3.2  in  Polak  (1997),  which  we  state  here  without  proof  for  the  sake  of 
completeness. 

Proposition  III. 14.  The  epigraphs  EN,  N  G  A f ,  of  the  problems  (P/v)  converge  to 
the  epigraph  E  of  the  problem  (P)  if  and  only  if 

(a)  for  every  x  G  X,  there  exists  a  sequence  {a/ArlA/eA/-,  with  Xn  G  Xn,  such  that 
xn  ->a/”  x,  as  N  — *  cx),  and  lirnsup Jn(xn)  <  /(x);  and 

(b)  for  every  infinite  sequence  {a/ArlweA',  with  K  C  A/”,  such  that  Xn  G  Xat,  for  all 

A^  G  A',  and  a; at  — i,  as  iV  g  oo,  i  G  T  and  liminf  /w(^v)  >  /(x).  □ 

We  would  like  to  establish  epi-convergence  of  ( GTPn )  to  (GPP);  un¬ 
fortunately,  this  is  not  possible.  For  this  reason,  we  make  a  small  modification  to 
(■ GTP )  and  replace  H°  with  its  closure,  H°;.  We  define  this  new  problem  as  follows 

( GTPd )  min  (III. 88) 

ven°cl 

It  is  possible  to  establish  epi-convergence  of  (GTPn)  to  (GTPd).  We  use  the  problem 
(GTPd),  with  the  following  assumption. 

Assumption  III. 15.  We  assume  that  all  local  and  global  minimizers  of  (GTPd)  are 

in  H° .  □ 

Using  an  approach  similar  to  that  found  in  Section  3.3  of  Polak  (1997), 
we  next  show  that  the  pairs  ((GTPN),9N)  in  the  sequence  {((GTPn),0n)}n£M  are 
consistent  approximations  for  the  pair  ((GTPd) ,9),  which  ensures  that  globally  and 
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locally  optimal  points  as  well  as  stationary  points  of  ( GTPN )  converge  to  correspond¬ 
ing  points  of  ( GTPci ),  as  N  — *  oo.  In  order  to  show  the  epi-convergence  of  ( GTPjy ) 
to  (GTPci),  we  will  need  the  following  intermediate  result. 

Proposition  III. 16.  Suppose  that  Assumptions  III.l,  III. 2  and  III. 3  are  satisfied. 
Then,  given  a  bounded  subset  S  C  H,  there  exists  a  Ks  <  oo  such  that  for  every 
rj  G  S  fl  Hjv,  and  N  G  Af, 

\Mv)  -  m\  <  (III. 89) 

Proof.  For  any  r)  G  S  fl  Hjv  and  N  G  Af,  we  have  that 


\fN(v)~f(v)\  = 
< 


'  aeA 


exp(—z^(l]a))(f)(a)da—  /  exp  (— zv(l]  a))  cj)(a)da 


I  aeA 


' aeA 


|exp  (— ^(1;  a))  —  exp  (—zn(  1;  a))|  f>(a)da. 


(III. 90) 


By  Theorem  5.6.23  in  Polak  (1997),  we  deduce  that  we  can  bound  the  approximation 
error  1^(1;  a)  —  2^(1;  a)|.  Specifically,  for  any  aeA,  given  a  bounded  subset  ScH, 
for  every  77  G  S  fl  H/v,  and  N  G  Af,  we  have 

|«’'(lia)-4(l;a)|<^,  (III.91) 

with 

Ks  =  max  1 1,  J  +  Kf,  (III.92) 

and 

K%  —  K'  exp(K')  supdie'll  +  1  I  (£,u)eS},  (III.93) 


K  as  in  Assumption  III.3(vii),  and  K'  as  in  (III. 32).  Hence,  by  the  properties  of  the 
exponential  function  and  the  fact  that  znN(  1;  a)  >  0  for  all  77  G  Hu  and  zv(l;a)  >  0 
for  all  77  G  H|,  for  any  aeA,  and  every  77  G  S  fl  H/v,  and  N  G  Af, 

|exp(-4r(l;«))  —  exp  (—zv(l;  a))|  <  \zv(V,a)  -  zvN(l;a)\  <  (III.94) 

Because  Ks  does  not  depend  on  a,  we  have  that 

l/jvfa)  ~  f(v)  I  <  /  lexP  (--tU1;  «))  -  exP  (-^(!;  «))|  f(a)da  <  (III.95) 

Ja&A  -/V 

□ 


We  now  show  that  (GTPjy)  epi-converges  to  ( GTPci ). 
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Theorem  III. 17.  Suppose  that  Assumptions  III.l,  III. 2,  III.  3  and  III.  15  are  satis¬ 
fied.  Then  ( GTPn )  epi-converges  to  (GTPci),  as  N  — *  oo. 

Proof.  We  show  that  ( GTPn )  epi-converges  to  ( GTPci ),  as  N  — >  oo,  by  showing  that 
the  conditions  in  Proposition  III.  14  are  satisfied.  Let  rj  G  be  arbitrary.  Then, 
from  Proposition  III. 8,  there  exists  a  sequence  {vn}Ngat  such  that  t)n  £  H1/,  for  all 
N  G  A f,  and  t}n  — >A/”  rj,  as  AT  — y  oo.  Let  e  >  0.  By  the  H-continuity  of  /(•),  there 
exists  an  TV  G  A/  such  that  for  all  N  >  TV,  ^  <  |  and  (/(t/at)  —  /(a) I  <  §,  where  TVs 
is  as  in  (III. 92).  Hence,  by  Proposition  III.  16 

|/w(w)  -  f(v)\  <  \In(vn)  ~  f(VN)\  +  \f(VN)  ~  f{v)\ 

Ks  e 
<  —  +  - 
-  N  2 

<  e,  (III.96) 

for  all  N  >  N,  N  E  Al.  Consequently,  /v(^w)  — ^  /(??)  as  N  ^  oo,  satisfying 
condition  (a)  of  Proposition  III.  14. 

In  order  to  show  that  condition  (b)  of  Proposition  III.  14  is  satisfied, 
suppose  that  a  sequence  {t]n} n^m  is  su°h  that  r/jv  G  H(/  for  all  N  G  A/”,  and  t]n  — rj, 
as  At  — >  cx).  Then,  based  on  the  construction  of  H(/  in  (III. 58),  we  must  have  that 

rj  G  H/ .  It  again  follows  from  the  H-continuity  of  /(•)  and  Proposition  III. 16  that 

/a/Vv)  /O'?)  as  At  — >•  oo,  which  satisfies  condition  (b)  of  Proposition  III. 14.  This 
proves  that  ( GTP V)  epi-converges  to  ( GTPci ).  □ 

In  order  to  show  that  the  pairs  (( GTPn ),  On),  in  the  sequence 
{((GTPN),0N)}NeAr,  and  the  pairs  (( GTP £),  0^),  in  the  sequence  {((GTP^),  6^)}^^ 
are  consistent  approximations  for  the  pairs  (( GTPd),0 )  and  (( GTPC),6C ),  respec¬ 
tively,  we  will  need  the  following  two  results. 

Proposition  III. 18.  Suppose  that  Assumptions  III.l,  III. 2  and  III. 3  are  satisfied, 
then  for  every  bounded  subset  S  C  H°?  there  exists  a  constant  Kf  <  oo  such  that,  for 
any  N  G  J\f  and  rj  G  S  D  Hn 

l|VWi))-V/(i,)||ffl<^.  (III.97) 
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Proof.  Based  on  the  definitions  of  f(rj)  and  /at(?/)  given  in  (III. 14)  and  (III. 62), 
respectively,  and  Propositions  III. 5  and  III. 11, 


>  aeA 


VvfN(rj',  a)(f>(a)da  —  /  V vf (v,  a)(j>(a)da 


'  a£A 


H  2 


< 


I  aeA 


VnfN(ir,  a)  -  a)  (f)(a)da. 


\h2 


(III. 98) 


We  now  focus  on 


V,;/at(? ?;  «)  —  Vr?/(r7;ci;)  .  By  Theorem  5.6.26  in  Polak  (1997), 


\h2 


for  any  a  e  A,  and  for  every  bounded  subset  S  C  H°.  there  exists  a  AV  <  oo  such 
that,  for  all  N  7  N,  and  p  e  S  D  Hn, 

Kf 


Vf//jv(h;«)  -  V„/(r7;o) 


< 

h2  N 


(III. 99) 


We  deduce  from  the  proof  of  Theorem  5.6.26  in  Polak  (1997)  and  Assumption  III. 3 
that  Kp  is  independent  of  a.  Then,  we  have  that 


V,;/v(h; «) 


V, ,/(?/;  a) 


4>(a)da  < 

#2 


(III.100) 


which  completes  the  proof.  □ 

Because  we  can  only  establish  epi-convergence  of  (GTP/v)  to  ( GTPci ),  and  not  (GTPjy) 
to  ( GTP ),  we  will  need  the  following  intermediate  result. 

Lemma  III. 19.  Suppose  denotes  the  closure  of  H".  If  a  function  V/3  :  H°  — >  H 
is  Lipschitz  H -continuous  on  bounded  subsets  of  H°;  then  it  is  also  Lipschitz  H- 
continuous  on  bounded  subsets  of  . 


Proof.  Because  H"  is  constructed  from  H  by  choosing  7  e  (0, 1)  (see  (III. 7)  and 
(III. 8)),  it  is  always  possible  to  construct  an  H(,/  that  is  slightly  larger  than  H11  by 
choosing  a  7  closer  to  one.  Since  H1  contains  H(),  and  it  can  be  shown  that  V/3(-) 
is  Lipschitz  H-continuous  on  bounded  subsets  of  Hlj/,  we  can  conclude  that  V/3(-)  is 
Lipschitz  H-continuous  on  bounded  subsets  of  H°z.  □ 

We  now  show  that  the  pairs  (( GTPn),9n )  in  the  sequence 
{((GTP/v),  0]\r)}NeJV  are  consistent  approximations  for  the  pair  (( GTPci),9 ). 
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Theorem  III. 20.  Suppose  that  Assumptions  III.l,  III. 2,  III. 3,  and  III. 15  are  sat¬ 
isfied,  and  that  ( GTPci ),  9,  ( GTPn ),  and  On  are  defined  as  in  (III. 88),  (III. 51), 
(III. 86),  and  (III. 83),  respectively.  Then,  the  pairs  ((GTPn),  On),  in  the  sequence 
{((GTPn),On)}n£AT  are  consistent  approximations  for  the  pair  ((GTPci),  0). 

Proof.  Suppose  that  an  infinite  sequence  {^jvjjv&V  is  such  that  t]n  £  H^,  for  all 
TV  e  A f,  and  t]n  V,  as  TV  — y  oo.  Let  e  >  0.  From  Lemma  III.6  and  Lemma  III.  19 

we  know  that  V/(-)  is  Lipschitz  H-continuous  on  bounded  subsets  of  H^.  By  the 
H-continuity  of  V/(-),  there  exists  an  TV  e  N  such  that  for  all  TV  >  TV,  ^  <  |  and 
||  V/ (vjn)  —  Vy (77) ||  <  |,  where  Kp  is  as  in  Proposition  III. 18.  Hence,  by  Proposition 
III. 18 

HV/jvfajv)  -  V/(t7)||H2  <  ||V/iVM-V/M||^  +  ||V/M-V/(r7)|U2 

KF  e 
<  —  +  - 
“  TV  2 

<  e,  (III.101) 

for  all  TV  >  TV,  TV  e  J\f .  Consequently,  V/w(w)  ^  V/( 77),  as  TV  — >■  00,  implying 
that  On(vn)  — ^  0(rj),  as  TV  — y  00.  Theorem  III. 17,  together  with  the  convergence  of 
0N (tin)  to  0(rf)  as  TV  — y  00,  satisfies  the  requirements  of  Definition  III. 2  for  consistency 
of  approximation.  □ 

Next,  we  show  that  the  pairs  ((GTPff)  >  @n)  in  the  sequence 
{((GTPcN),  On)}n£ M  are  consistent  approximations  for  the  pair  ((GTPC),0C). 

Theorem  III. 21.  Suppose  that  Assumptions  III.l,  III. 2,  III. 3,  and  III.  15  are  sat¬ 
isfied,  and  that  (GTPC),  0C ,  (GTPff),  and  0CN  are  defined  as  in  (III. 20),  (III. 52), 
(III.  66),  and  (III.  84),  respectively.  Then,  the  pairs  ((GTPffi),  0CN),  in  the  sequence 
{{(GTPcn),  Ocn)}n&m,  are  consistent  approximations  for  the  pair  ((GTPC),0C). 

Proof.  The  proof  that  the  problems  (GTPff)  epi-converge  to  (GTPC)  is  the  same  as 
the  proof  of  epi-convergence  of  the  problems  (GTPn)  to  (GTPci)  given  in  Theorem 
III.  17  above. 

From  the  proof  of  Theorem  III. 20  above,  we  know  that  V/at(t1/v)  — Pv' 
V  f(r]),  as  TV  — y  00.  Then,  following  the  same  arguments  as  in  the  proof  of  Theorem 
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such  that 


4.3.6  in  Polak  (1997)  we  see  that  given  any  infinite  sequence  {vn}n&j^, 
rjN  £  Hc.n  for  all  N  £  Al,  which  converges  to  an  rj  £  Hc,  9cn(t}n)  — 9c(q),  as  N  — * 
oo.  Theorem  III. 20,  together  with  the  convergence  of  Of  (tin)  to  9c(r /)  as  iV  — >  oo, 
satisfies  the  requirements  of  Definition  III. 2  for  consistency  of  approximation.  □ 

b.  Time-  and  Space- Discretized  Problems 

We  next  consider  the  time-  and  space-discretized  problem.  As  discussed 
in  Section  II. A  we  focus  on  A  C  M2,  and  introduce  the  space  discretization  parameter, 
M  =  (Mi,  M2)T  £  N  x  N.  We  also  define  a  generic  numerical  integration  scheme,  Im, 
for  a  function,  T  :  CP(A )  — >  M,  where  Cp  represents  the  differentiability  class.3  The 
integration  scheme  Im  is  a  mapping  from  CP(A )  to  M  or  to  Mn+m  for  any  M  £  N  x  N 
and  T  £  CP(A )  defined  by 

Mi  M2 

*«(»)  = 

i=i  j= 1 

where  Ilfij,  i  =  1,2,  j  =  1,2,  ...,M2,  are  weights  for  the  chosen  numerical 

integration  scheme,  and  atl  are  the  discretization  points  at  which  the  integrand  is 
evaluated. 

We  then  make  use  of  the  integration  rule,  Im,  to  define  the  approximate 
objective  function  f^M  ■  — y  M  for  any  rj  £  H^,  N  £  A f,  and  M  6  N  x  N  by 

fNAfiv)  ~  (exp  [-zvN(  1;  •)]  </>(•))  •  (III.103) 

We  next  consider  the  differentiability  of  /atm(-)- 

Proposition  III. 22.  Suppose  that  Assumptions  III.l,  III. 2,  and  III. 3  are  satisfied, 
Im  is  defined  as  in  (III.  102),  N  £  A f ,  Me  Nx  N,  and  /mm  '■  — »  M  is  defined  as 

in  (III.  103),  then  for  any  q  e  H°N  and  5q  £  H0 0j2,  /jvm(')  a  Gateaux  differential 
DfNM(v,dq)  =  (yfNM(jl),Sq)H^  where 

VfNM(v)(t)  =  Im  [^Jn(v;  •) (t)0(-)]  ,  Vt  £  [0, 1].  (III.104) 

3The  class  C°  consists  of  all  continuous  functions  on  A.  For  any  positive  integer,  p,  Cp  is  the  set 
of  all  differentiable  functions  on  A  whose  gradient  is  in  Cp_1. 


ty(a)da, 


(III. 102) 
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Proof.  Let  Sr)  E  H 00,2,  and  r)  E  H^r  be  arbitrary.  From  Lemma  111.10(a)  we  know 


DfN(rr  or,  Sr))  =  (  VvfN(i T  «),  Sr) 


h2 


(III.105) 


Then,  we  have 


DfNAdij)'i  Srj)  = 


f nm (j)  +  A< fry;  a)  -  fNM{m  a) 


A 


In{j)  +  A  Sr)-,  Oij )  -  /jv(r 7;  a 


lim 

A4.0 

Mi  M2 

'isEE1^ 

i=l  i=l 

Mi  M2 

E  Ew«  “s 

i= 1  1=1 

Mi  M2 

E  Oij),  S-q)  (^OLij) 

11  '  ' 

1=1  j=\ 

Mi  M2 

EEW«V  T7^JV ( ^7 5  ^ij)4^ip^ij)  1  dg 

i=l  3  =  1  /  h2 


(III. 106) 


Our  next  result  is  related  to  the  Lipschitz  Hjv-continuity  of  V/atm(^) 
on  bounded  subsets  of  H^. 

Lemma  III. 23.  Suppose  that  Assumptions  III.l,  III. 2,  and  III. 3  are  satisfied,  then 
the  gradient  V /nm (j))  is  Lipschitz  H ^-continuous  on  bounded  subsets  ofH()N. 


Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Lemma  III.  12,  with 
integration  replaced  by  Im-  □ 

In  the  analysis  that  follows,  it  is  necessary  to  quantify  the  error  in¬ 
troduced  by  the  numerical  integration  scheme  in  order  to  complete  the  convergence 
proofs.  Clearly,  the  choice  of  numerical  integration  scheme  determines  the  relationship 
between  the  discretization  level  and  the  amount  of  error  introduced  by  the  approxi¬ 
mation.  In  order  to  conduct  our  analysis,  we  make  the  following  assumptions. 
Assumption  III. 24.  We  assume  that  there  exist  scalars  a,b,c,  and  d  such  that 

(i)  A  =  {(ai,a2)  E  M2|a  <  aq  <  b,  c  <  a2  <  d},  where  A  represents  the  region  of 
integration  in  Im- 
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(ii)  aij  =  (ay (i),a2(j))T  and  an  integer  Mi  is  chosen  so  that  the  interval  [a,  b]  is 
subdivided  into  2M\  subintervals  {[«i(i  —  1), Q'i(i)]}^1  of  equal  width  h  = 
(■ b  —  a)/2Mi  by  using  the  equally  spaced  discretization  points  oq(i)  =  a  +  ih 
for  i  =  0, 1, 2, 2M\.  An  integer  M2  is  chosen  so  that  the  interval  [c,d]  is 
subdivided  into  2 M2  subintervals  { [a2 (j  —  1),  «2(j)]}j=i2  of  equal  width  k  = 
(d  —  c)/2M2  by  using  the  equally  spaced  discretization  points  a2(j)  =  c  +  jh  for 
j  =  0, 1,2,  ...,2 M2.  Composite  Simpson’s  rule  is  used  for  numerical  integration 
and  for  'h  G  CP(A)  is  given  by 


T  ,  T  N  hk 

ImW  -  — 


m2 


T(a,  c)  +  T(a,  d )  +  T( b,c )  +  T (b,  d)  +  4  ^  T(a,  a2(2j  -  1)) 

i=i 

M2  —  1  M2  M2  —  1 

2  ^  d/(a,a2(2j))+4^vI;(6?«2(2J-l))  +  2  4>(6,a2(2j)) 

j=i  i=i  i=i 

Mi  Mi-1  Mi 

4  'h(o;i(2?'  —  1),  c)  +  2  T(aq(2i),  c)  +  4  T(ai(2i  —  1),  <f) 

i=  1  2=1  2—  1 

Mp  — 1  M2  Mi 

2  ]T  ^(a1(2^),d)  +  16^^^(a1(2*-l),a2(2j-l)) 

2—1  J  =  1  2—1 

M2-l  Mi  M2  Mi-1 

*ee  vh(a1(2i  —  1),  a2(2j))  +  8  EE  'B(a1(2i),a2(2j  -  1)) 

j=l  i= 1  j=l  *=1 

M2-IM1-I  \ 

*E  e  4/(o;1(2i),  a2(2j))  )  (III.107) 

1=1  *=1  / 


□ 


We  note  that  the  convergence  proofs  given  below  in  Proposition  III. 26  would  follow 
similar  arguments  if  A  had  higher  dimensionality,  or  a  different  numerical  integration 
scheme  had  been  utilized.  The  proofs  could  also  be  done  if  A  was  assumed  to  be  a 
shape  other  than  rectangular,  but  they  would  be  more  complicated. 

We  find  it  necessary  to  show  that  the  partial  derivatives  of  /v(dj  •)</>(•) 
and  Vf//jv(?7;  -)0(-)  up  to  and  including  the  fourth-order  are  bounded  for  any  77  G  H^, 
a  G  A,  and  N  G  M  in  order  to  complete  the  proofs  of  convergence  of  /vm(^)  to 
f(rj)  and  V /vm(^)  to  V/(r/),  based  on  the  choice  of  Composite  Simpson’s  rule  as  the 
numerical  integration  scheme.  To  facilitate  these  proofs,  we  begin  by  defining  some 
notation.  For  any  rj  G  Hi,  a  G  A,  N  G  A /”,  and  j  =  0, 1, N  —  1,  we  define 
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Cs(«)=^(0;a),  (III.110) 

and 

C4(a)  =  Pn  (^-'i  •  (III.lll) 

We  note  that  Ci(')>  Cs(')>  and  C4(')  depend  on  r/  and  N.  We  next  show  that  the  partial 
derivatives  of  Ci(')>  •••>C4(')  up  to  and  including  the  fourth  order  are  continuous  and 
bounded  for  any  p  G  H%,  a  G  A,  and  N  G  AT. 

Lemma  III. 25.  Suppose  that  Assumptions  III.l,  III. 2,  and  III. 3  are  satisfied  and  S 
is  a  bounded  subset  ofH()N.  Then, 

(i)  U-)eC\A),i  =  1,2, 3, 4, 
and 

(ii)  there  exists  C  <  oo,  such  that  for  all  p  G  S,  j  =  0, 1,  ...,N  —  1,  a  G  A,  and 
N  G  Af 

<C  V*  =  1,  2,  V/i  =  1,  2,  3, 4,  Vk  =  1,  2,  3, 4.  (III.112) 

Proof.  It  can  be  seen  by  repeated  applications  of  the  chain  and  product  rules  that 
based  on  Assumptions  III.2(i)  and  III.3(iii),  Ci(')  G  C4(A).  We  also  know  from  As¬ 
sumption  III.l  that  C2(‘)  ^  C4(A). 

We  now  consider  £3(o).  We  know  that  pnN{ l;a)  G  C4(A)  for  the  same 
reasons  that  Ci(-)  G  C4(A),  because pnN(l]  a)  is  a  column  vector  of  nK  zeros  and  —  Ci(-)- 
We  also  know  that  hx  (xvN(^),u(^)-,a),  j  =  0, 1,  ...,N-1,  has  hkx  (xvN{jj),u{jj))  ,  k  = 
1,  2,  ...,K,  and  £f=1  £f=1  (  xn( n)i  yl(  am  °0)  as  components.  Because  there  is 

no  a  dependence,  hk  and  partial  derivatives  of  hk  (xvN(jj),  with 


da1- 
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respect  to  a  are  constants  for  all  k  —  1,2,  Based  on  Assumptions  III.2(i)  and 

III.3(iv),  Vxrk’1  •))  G  C4(A).  This  means  that  hx  {xvN(jj),  ■)  G 

C4(A).  Next,  we  proceed  with  an  induction  argument.  Suppose  that  pr)N((j+l)/N\  •)  G 
C4(A),  then  pvN(j/N;  •),  given  by 

PnU/n>  •)  =  p^((i+1)/Ar; wO'/Af)}  •)Tp^((i+1)/Ar;  ■)»  (ni.113) 


is  also  C4(A).  Hence,  it  then  follows  by  induction  that  pnN( 0;  •)  G  C4(A),  and  therefore 

0.(0  G  C4(A). 

By  the  induction  argument  above,  we  know  that  pvN( •)  G  C4(A)  for 
all  j  —  0, 1, ...,  N  —  2.  Hence,  £4(-)  G  C4(A),  and  the  proof  of  part  (i)  is  complete. 

We  start  the  proof  of  part  (ii)  by  looking  at  Ci(°0-  If  can  be  seen  by 
repeated  use  of  the  product  and  chain  rules  that  the  first  through  fourth-order  partial 
derivatives  of  are  made  up  of  sums,  differences,  and  products  of  the  expressions 

»p  [-£  (A  (£)  V  (l?;“))l  (in. n4) 

.  j= 0  fc=l  (=1  v  v  7  v  7  7  _ 

and 


V*  =  1,2,Vk  =  1,2,3,4.  (III. 115) 


To  prove  that  the  first  through  fourth-order  partial  derivatives  of  Ci  (•)  are  bounded, 
and  that  the  bounds  are  independent  of  N  and  r],  we  show  that  expressions  (III.  114) 
and  (III.  115)  are  bounded  with  respect  to  a,  and  that  their  bounds  are  independent 
of  N  and  rj.  We  first  consider  (III. 114).  By  Assumption  III.3(i),  there  exists  Cr  <  oo 
such  that 

rkt  (*t  (A)  J  |  <  Cr,  (III. 116) 
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for  all  77  G  S,  k  =  1,  2, K ,  l  =  1,  2, A,  a  G  A,  j  =  0, 1, N  —  1,  and  N  G  A f. 
Then 


N—l  n  K  L 


N 

j= 0  fc=i  2=1 
JV-1  I<  L 

*  vEEE 

j=0  fc=l  2=1 


AT 


2  / 


%  t m/ 


AT 


AT 


1 


JV-1 


<  —  V  KLCr  =  KLCr 
-  N  ^ 


(III. 117) 


i=o 


Hence,  (III.  114)  is  bounded  by  exp(— KLCr),  which  is  independent  of  N  and  77. 

Next,  we  consider  (III. 115).  By  Assumptions  111.2(h)  and  III.3(vi) 


dh 
da 7 


k,l  (  „v,k  (  l_ 


r  ’  \  xn 


i  A 


AT 


<  Cr\Cy , 


(III. 118) 


for  all  77  G  S',  k  =  1,  2, ...,  A',  l  =  1,  2, ...,  A,  a  G  A,  7  =  1,  2,  j  =  0, 1, ...,  N  —  1, 
k  =  1,2, 3, 4,  and  N  G  A/”,  with  Cr\  and  Cy  as  in  Assumptions  111.2(h)  and  III.3(vi), 
respectively.  Then,  for  all  7  =  1,2,  and  k  =  1,  2,  3, 4 


N—l  ^  K  L 

E^EE^/m,* 


N  daf 

j=0  1  k= 1  2=1 


i(L.a 

N' 


(III. 119) 


< 


< 


N- 1  A'  L 

vEEE 

j= 0  fc=l  2=1 

1 

-  ^  KLCrlCy  =  KLCrlCy. 


(III. 120) 


3=0 

Therefore,  (III.  115)  is  bounded  by  KLCriCy,  which  is  independent  of  N  and  77.  Be¬ 
cause  (III. 114)  and  (III. 115)  are  bounded,  and  their  bounds  are  independent  of  N 
and  77,  the  first-  through  fourth-order  partial  derivatives  of  Ci(oO  with  respect  to  a 
are  bounded,  and  the  bounds  are  independent  of  N  and  77. 

Now,  we  consider  (2 (a).  From  Assumption  III.  1  and  the  compactness 
of  A,  we  know  that  the  first-  through  fourth-order  partial  derivatives  of  (2 (a;)  are 
bounded  for  any  a  G  A. 
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We  next  consider  £3(a).  We  know  that  the  first-  through  fourth-order 
partial  derivatives  of  pvN(l]a)  with  respect  to  a,  are  bounded  for  the  same  reasons 
that  the  first-  through  fourth-order  partial  derivatives  of  (i(o;)  are  bounded.  The 
first-  through  fourth-order  partial  derivatives  of  hkr  (x^(-),u(-)),  k  =  1,2,  with 

respect  to  a  are  bounded  because  they  are  all  equal  to  zero.  The  first-  through  fourth- 
order  partial  derivatives  of  \7xrk’1  (yxvN(t),yl(t]  a))  with  respect  to  a  are  bounded  for 
any  t  £  [0, 1]  by  Assumptions  III.2(ii)  and  III.3(vi),  and  the  bounds  are  independent 
of  t  and  N.  Then,  because  hx(x(t),u(t );  a)  is  earlier  defined  by 

(  h1x(x(t),u(t))T  ^ 

hx(x(t),u(t);a)  =  ,  ,  (III. 121) 

hx(x(t),u(t))T 

(LLi  Eti  Vxrk’l(xk(t),yl(t;  a))ry 

there  exists  K'~  <  oo  such  that 

<  n,.  (IIL122) 

for  all  r)  £  S,  a  e  A,  i  —  1, 2,  j  —  0, 1,...,7V  —  1,  k  =  1,2, 3, 4,  and  N  e  A f. 
From  Assumptions  III.3 (ii) ,  (iii),  and  (v),  we  know  that  hx  {xvN(j /N),u(j /N);a)  is 
continuous  with  respect  to  a  for  all  rj  G  S,  on  A.  Since  hx  (xvN(j / N) ,  u(j /N)]  a)  is 
continuous  with  respect  to  a,  and  S  is  a  bounded  subset  of  H^,  there  exists  <  oo 
such  that 

\hx{x%{j/N),u{j/N)-a)\\  <  K~hx ,  (III. 123) 

for  all  r)  £  S,  a  £  A,  j  =  0, 1, ...,  N  —  1,  and  N  £  J\f .  Because  zj^(  1;  a)  >  0  for  all 
rj  £  S  and  a  £  A,  there  exists  C\  <  oo  such  that  ||p^(l;o;)||  <  C\  for  all  rj  £  S, 
a  £  A,  and  N  £  A f.  To  show  that  pvN(j/N ;  a)  is  bounded  for  all  rj  £  S,  a  £  A,  and 
j  =  0, 1, ...,  AI  —  1,  we  proceed  with  an  induction  argument.  Suppose  that  there  exists 
Cj+ 1  <  oo,  such  that  || pvN((j  +  1  )/N]  a)||  <  Cj+ 1,  for  all  r)  £  S,  a  £  A,  and  N  £  A f. 
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Then,  based  on  (III.  113) 


\WU/N-,<*)\\  <  \K((j  +  l)/N;a) 

hx(x%(j/N),u(j/N);  a)TpvN((j  +  1  )/N;  a) 


N 


<  cj+1  +  —Ki  a--M  =  c 


N 


fc^l+l  Lyl+1 


1  + 


Kr 


N 


(III. 124) 


By  doing  another  step  in  the  backward  recursion,  we  find  that 


\\pl((j-l)/N;a)\\  <  Cj+1  1  + 


Kr 


N 


+ 


Kr 


N 


-C. 


l+i 


1  + 


Kr 


N 


=  C 


l+i 


-  Cj'+i 


1  + 


Kh  '  2 

'Lx 

~w 


1  + 


Kr 


N 


N 


(III. 125) 


There  exists  N  e  N  such  that  for  all  N  >  N,  ^1  +  <  2eK~h* .  Then,  for  any 

N>N , 

I \pnM  ~  l)/iV;«)||  <  2 Cj+1eK>*.  (111.126) 

For  values  of  N  smaller  than  N,  we  define 

/  k~  \ 

,  *  =  1,2,...,1V-1,  (III. 127) 

and 

K$  =  max{  max  {<5j},2}.  (III. 128) 

i=l,2,...,JV-l 

Then, 

WpKU  -  l)/JV;a)||  <  KsCj+1eK-..  (III.129) 


Hence,  it  follows  by  induction  that  pvN(j/N]a )  is  bounded  for  all  p  e  S,  a  e  A, 
j  =  0, 1, ...,  N  —  1,  and  the  bound  is  independent  of  N  and  p. 

We  now  proceed  with  another  induction  argument  to  show  that  the 
first-  through  fourth-order  partial  derivatives  of  pvN(j /N]  a)  with  respect  to  a  are 
bounded.  Suppose  the  first-  through  fourth-order  partial  derivatives  of  pnN((j  + 
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1  )/N\a)  with  respect  to  a  are  bounded  and  that  the  bound,  C')+1  <  oo,  is  inde¬ 
pendent  of  N  and  77.  Then,  based  on  (III. 113),  for  all  77  G  S,  a  G  A,  i  =  1, 2, 
j  =  0, 1, ...,  AT  —  1,  k  =  1,  2,  3, 4,  and  N  £  J\f 


dK 


1 

'N 


daK 

dK 


PNti/N',a) 


< 


dK 


:Pn((J  +  i)/-W; «) 


da f 


dal ‘ 

hx(xvN(j/N),u(j/N);  a)TpvN((j  +  1  )/N;  a] 


<  cH1  +  v 


1 

N 


dK 


hx(xvN(j/N),u(j/N);a)T-^pvN((j  +  1  )/N\a 


hx(xvN(j/N),u(j/N);  a)T  pvN((j  +  1)/N;  a 


<  c'j+1  + 


-  Cj+ 1  1  1  + 


dao 

K~hC’j+1  +  Cj+1Kl 


N 


Ki 


+ 


K'~hCj+i 


N  J  N 

By  doing  another  step  in  the  backward  recursion,  we  find 

d 


(III. 130) 


daf 


Pn(U  -  1)/N~> «) 


—  Cj+ 1  I  1  + 


—  Cj+ 1  f  1  + 


Ah  \2  KLCj+1  _  K~hK>~Cj+l  CjKL 


N 


+ 


N 


+ 


N2 


+ 


N 


Ki  ^ N  KLCj+1  K-hK'-Cj+1  CjK'r 


N 


+ 


N 


+ 


N 2 


+ 


<  KsC'+1eK^  +  Kc, 


N 

(III. 131) 


where  Kc  <  00  is  such  that 


K'~  Cj+1  Kh  K'~  Cj+1  CjKL 

TS-  \  hx  ■'T-l  ,  nx  hx  JT1  .  J  hx 

A  C  >  - 77 - t - 777 - h 


AT2 


AT 


(III. 132) 


for  any  A"  e  A/".  Hence,  it  follows  by  induction  that  the  first-  through  fourth-order 
partial  derivatives  of  pnN{ 0;  •)  with  respect  to  a  are  bounded,  and  therefore  the  first- 
through  fourth-order  partial  derivatives  of  Q$  ( • )  with  respect  to  a  are  bounded.  Fur¬ 
thermore,  these  bounds  are  independent  of  N  and  77. 

By  the  induction  argument  above,  the  first-  through  fourth-order  par¬ 
tial  derivatives  of  pnN( •)  with  respect  to  a  are  bounded  for  all  j  —  0, 1, ...,  N  —  2 
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and  the  bounds  are  independent  of  N  and  77.  This  means  that  the  first-  through 
fourth-order  partial  derivatives  of  £4(-)  with  respect  to  a  are  bounded,  and  the  proof 
of  part  (ii)  is  complete.  □ 

In  order  to  prove  epi-convergence  and  consistency  of  approximation,  we 
need  the  following  proposition. 


Proposition  III. 26.  Suppose  that  Assumptions  III.l,  III. 2,  III. 3,  and  III. 24  are 
satisfied.  Then  for  every  bounded  subset  S  C  H°,  there  exist  constants  Kj\ ,  K j2  <  00 
such  that,  for  any  N  e  A f ,  M  e  N3  x  N3,  where  N3  =  {m  e  2N  +  l\m  >  3},  and 

t)  g  s  n  Hn  , 


and 


I /(h)  -  Inm(v)  I  < 


Ks  Kn  Ki  2 

N  +  (Mi  -  l)4  (M2  -  l)4 


(III. 133) 


l|V/(h)-V/vM(h)lk< 


Kf  Kn  Kn 

N  +  (. M !  -  l)4  (M2  -  l)4’ 


(III. 134) 


and  Ks  and  KF  are  defined  as  in  (III.  92)  and  Proposition  III.  18,  respectively. 


Proof.  We  know  that 


|/(h)  -  f nm(j]) I  <  |/(h)  -  /iv(h)l  +  |/iv(h)  -  f nm(j]) I  ,  (III. 135) 


and 


||V/(h)-V/ATM(h)ll^  <  ||V/(h)-V/iv(h)||^+'||V/iv(h)-V/ivM(h)l|H2.  (HI. 136) 


Based  on  Proposition  III. 16  and  Proposition  III. 18,  respectively,  1/(77)  — /jv(h)l  < 
and  || V/(h)  —  V/W(r/)|U2  <  for  any  77  G  S'  fl  H^.  In  order  to  deal  with 
IMh)  -  /viu(h)l  and  ||V/jv(h)  -  V/jvM(h)ll  h2,  we  define  for  notational  simplicity 


and 


9i(a)  -  Ci(«)C2(«), 

(III. 137) 

92(a)  =  VfN(ir,a)(j)(a), 

(III. 138) 
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where  by  Lemma  III. 10  and  the  definitions  given  in  (III. 109),  (III. 110),  and  (III. Ill) 
we  can  write  out  the  components  of  (72(a)  as 

g2i(a)  =  V?/iv(r 7;  a)(p(a)  =  pvN( 0;  a)<j>(a)  =  (3(a)(2(a),  (III.139) 


and  for  t  e  [0, 1] 
02ut(a) 


'TV— 1 


=  Vu/jv(^;Q!)(t)0(a)  = 

^  Mr  (*)  >m(tv)) 


(  ^;a)  7rjvj(t) 
l=o 

T 

(  hi  (rrV  (1_\  „  (±\\\ 


</>(«) 


TV-1 

E 

l=o 


hu  M  (&)>“(£)) 
0 


vr7Vj(t)C4(a)C2(a).  (III. 140) 


Then,  by  Lemma  III.25(i),  (7i(-)  €  CA(A)  and  g2{')  £  C4(A).  Again,  for  notational 
simplicity,  we  define 


A,  4 

£2*  = 


Im(9i(-))~  /  <71  (a)  da 

J  aeA 


lM(g2£i(-))  ~  /  g-2^i(a)da 
J  aeA 


(111. 141) 

(111. 142) 


and 


E 


A_ 

2  utj  — 


lM(g2utj(-)) 


g2utj{ot)da  , 


(III. 143) 


where  g2^ (a)  is  the  ith  component  of  g2% (a)  and  g2utj(a)  is  the  jth  component  of 
g2ut(oi).  From  Lemma  III.25(ii)  for  p  —  1,  2 2 utj,  i  =  1,2, ...,  n/l  +  1,  t  G  [0, 1],  and 
j  =  1,  2, ...,  m,  there  exist  constants  C\,  C2  <  00  that  have  no  dependence  on  77,  a ,  or 
N  such  that 


max 

a&A 


d4gP(a) 

da\ 


<C\ 


(III. 144) 


and 


max 

aeA 


d4gP(a) 

dal 


<C2. 


(III. 145) 
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Then,  under  Assumption  III. 24,  when  using  Composite  Simpson’s  rule  to  approximate 
the  integral  of  gi(-),  E\  is  bounded  by  (see,  for  example,  pages  127-128  in  Faires  & 
Burden,  1993) 


(b  —  a)4  d4g2^(a)  (d  -  c)4  d4g2ii(a) 

(Mi  -  1)  °6A  daf  (M2  -  1)  «eA  da\ 

(d  -  c)(b  -  a)  (b  -  a)4  (d  -  c )4  <  Kn  KI2 

180  [(Mi  -  l)4  1  (M2  -  l)4  '2J  “  (Mi  -  l)4  +  (M2  -  l)4’ 


E2  &  < 


(d  —  c)(b  —  a) 
180 


where  Kn  and  Kj2  are  as  above.  Similarly,  under  Assumption  III. 24,  when  using 
Composite  Simpson’s  rule  to  approximate  the  integral  of  g2utj(-),  E2utj  is  bounded  for 
t  G  [0, 1]  and  j  =  1,  2, ...,  m  by 


,  (d-c)(b-a)  [  (b  —  a)4  d4g2ut(a)  (d  -  c)4  d4g2ut(a) 

J  ~  180  [(Mi-1)4  oceA  daf  (M2  -  1)  daf 

<  (d  —  c)(b  —  a)  r  (b-a)4  (d  -  c)4  1  <  Kn  KI2 

180  [(Mi  -  l)4  1  (M2  -  l)4  ”2J  “  (Mi  -  l)4  (M2  -  l)4’ 
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where,  again,  Kj\  and  Kj2  are  as  above.  Then,  we  have 


< 


IIV/wM  -  V/™(,)  fH2 

l|V£/K(r;)-V{/NM(r;)||2+  [  ||  V„/K(,)(()  -  V„/™(I,)(i)l|2* 

Jo 

f  V2  |  V-  (  kn  kn  V 

h  7Mi  - x)4  (M^  -  i)4/  ^  V(Mi  - x)4  (m2  -  i)V 


(nK  +  1  +  m) 


K 


n 


K 


12 


(Mi  -  l)4  +  (M2  -  1) 


(III. 148) 


We  let  Kj\  =  \JnK  + 1  +  mKn  and  AT 2  =  \JnK  4- 1  +  rriK j2 ,  and  (III.  134)  follows 
from  (III.  136).  Finally, 


AT  1  AT2  ^  ATi  KI2 

~  (Mi  -  l)4  +  (M2  -  l)4  “  (Mi  -  l)4  +  (M2-l)4’ 

(III. 149) 

and  (III.  133)  follows  from  (III.  135),  which  completes  the  proof.  □ 

For  any  IV  e  A/",  and  M  GNxN,  we  define  the  following  approximating 

problems 


(GTPmm) 

min 

(III. 150) 

2?ghT 

(GTPcnm) 

min 

jjGHc,jv 

(III. 151) 

For  any  N  e  A/",  and  tf  6  N  x  I,  we  define  nonpositive  optimality 
functions  (Tvm  :  H(y  — »  M  and  :  HCiat  — >  M  by 


Im  (<7i(-))  ~  /  gi(a)da 


'  a£A 


Onm(v)  —  ^2  IIWwm(^)  ||  tf2, 


(III. 152) 


and 

0Wfa)  -  nhn  (VfNM(v),v'  ~  v)h2  +  J  IIV  -  hlli2 ,  (III.153) 

which  characterize  stationary  points  of  ( GTPnm )  and  (GTP^M),  respectively. 

Proposition  III. 27.  Suppose  that  Assumptions  III.l,  III. 2,  III. 3,  and  III. 24  are 
satisfied. 
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(a)  0NM(-)  and  0CNM(-)  are  H^-continuous  functions. 

(b)  If  fi  G  H”v  is  a  local  minimizer  of  ( GTPnm ),  then  Onm(v)  —  0- 

(c)  If  fj  G  HCjjv  is  a  local  minimizer  of  (GTPf?M),  then  0%M(f) )  =  0. 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Proposition  1.1.6  in 
Polak  (1997),  with  the  norms  and  inner  products  replaced  by  their  H-2  equivalents.  □ 
The  proofs  of  convergence  that  follow  require  that  we  have  only  one 
discretization  parameter,  therefore  for  %  =  1,2,  we  define  M,  :  J\f  — »  N  and  make  the 
following  assumption. 

Assumption  III. 28.  We  assume  for  i  —  1,  2  that  M^(N )  — *  oo,  as  N  — y  oo.  □ 

It  is  not  possible  to  establish  epi- convergence  of  ( GTPNM^N f)  to  ( GTP ), 
but  it  is  possible  to  establish  epi-convergence  of  ( GTPNM{n ))  to  ( GTPd ).  We  now 
show  that  the  family  {((GTPnm(n)),  0nm(n))}n<=N  is  a  sequence  of  consistent  ap¬ 
proximations  for  (( GTPci),6 )  by  showing  that  the  conditions  of  Definition  III. 2  are 
satisfied. 

Theorem  III. 29.  Suppose  that  Assumptions  III.l,  III. 2,  III. 3,  III.  15,  and  III. 28  are 
satisfied,  ( GTPd ),  6,  (GTPNM^N)),  and  dNM ^  are  defined  as  in  (III. 88),  (III. 51), 
(III.  150),  and  (III.  152),  respectively.  Then  the  pairs  {(GTPNM^Nf),  &nm(n)),  in 
the  sequence  { ( (GT P nm(n)  ) ,  d nm(n)  ) } ngat  are  consistent  approximations  for  the  pair 
(( GTPd),0 ). 

Proof.  We  first  show  that  (GTPnm(n))  epi-converges  to  (GTPd),  as  N  — »  oo,  by 
showing  that  the  conditions  in  Proposition  III.  14  are  satisfied.  We  begin  by  showing 
that  part  (a)  of  Proposition  III.  14  is  satisfied.  Let  r/  G  be  arbitrary.  Then  from 
Proposition  III. 8  there  exists  a  sequence  {'On}  NGAr  such  that  pN  £  H^,  for  all  N  G  A f, 
and  on  — 0  as  N  — y  oo.  Let  e  >  0.  By  Assumption  III. 28  and  the  H-continuity 
of  /(•),  there  exists  an  N  G  Af  such  that  for  all  N  >  N,  +  (Myv)2-!)4  —  h 

<  |,  and  \f(oN)  ~  f(v) I  ^  h  where  Kn  and  KI2  are  as  in  Proposition  III. 26,  and 
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Kg  is  as  in  (III. 92).  Hence,  by  Proposition  III.26(i), 


|  f nm{n)  (jin')  -  f(v)  |  <  |/jvM(iv)(r7iv)  -  f(m)  |  +  I  J(vn)  ~  f(v)  I 

Kn  KI2  e 

~  N  +  (Mi(iV)  -  l)4  +  (M2(N)  -  l)4  +  4 
<  e  (III. 154) 


for  all  IV  >  IV,  N  G  A/”.  Consequently,  f nm(n) (jin')  — ^  /(^),  as  IV  — *  oo,  which 
completes  the  proof  of  condition  (a)  of  Proposition  III.  14. 

In  order  to  show  that  condition  (b)  of  Proposition  III.  14  is  satisfied, 
suppose  that  a  sequence  {r)N}NeAf  is  such  that  t/n  G  H%  for  all  N  G  A/",  and  t/jv  — r/, 
as  IV  — *  oo.  Based  on  the  construction  of  H^,  we  must  have  that  r]  G  H” .  It  again 
follows  from  the  H-continuity  of  /(•)  and  Proposition  III.26(i)  that  f nm(N) (Vn)  — 
f{j])  as  N  — >■  tx),  which  satisfies  condition  (b)  of  Proposition  III. 14.  This  proves  that 


(GTPAtm( at))  epi-converges  to  ( GTPd ). 

For  the  second  part  of  the  proof,  we  show  the  convergence  of  the  opti¬ 
mality  functions.  Suppose  that  an  infinite  sequence  is  such  that  r//v  G  H^, 

for  all  N  G  A/",  and  t]n  — rj  as  N  —>  oo.  Let  e  >  0.  From  Lemma  III. 6  and  Lemma 
III. 19  we  know  that  V/(-)  is  Lipschitz  H-continuous  on  bounded  subsets  of  H^.  By 
Assumption  III. 28  and  the  H-continuity  of  V/(-),  there  exists  an  N  G  J\f  such  that 


for  a"  N  S  N, 


<  if  <  j  and  l|V/l>v)-V/(,,)||„,  < 


—  -  _  -  >  (M1(V)— 1)  1  (M2(AT)-1)4  -  2’  AT  -  4  II  v  •/  V'uvy  "  J  VU  im2  -  4’ 

where  Kn  and  Kn  are  as  in  Proposition  III. 26,  and  Kp  is  as  in  Proposition  III. 18. 


Hence,  by  Proposition  111.26(h), 


||  V/wM(iV)  (j]n)  -  V /  (77)  || 

<  II  VfNM(N)(VN)  -  V/(77jv)||^2  +  II V/M  -  V/(77)|U2 

Kf  Kn  KI2  e 

~  N  +  (Mi (IV)  -  l)4  +  (M2(N)  -  l)4  +  4 

<  e,  (III. 155) 


for  all  IV  >  IV,  IV  G  A/”.  Consequently,  V/jvm(at)(^v)  — f^  V/(?/)  as  IV  — >■  00,  and 
therefore  0nm(n)(vn)  — F^  0{rj)  as  IV  — >■  00.  The  epi-convergence  of  ( GTPnm(n ))  to 
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( GTPci )  as  N  — >  oo,  together  with  the  convergence  of  0NM(NfiriN)  to  6{rj)  as  N  — >  oo, 

satisfy  the  requirements  of  Definition  III. 2  for  consistency  of  approximation,  which 

completes  the  proof.  □ 

We  now  show  that  the  pairs  {{GTP^M^N^),6CNM^)  in  the  sequence 

Qcnm(n))}n£AT  are  consistent  approximations  for  the  pair  ((GTPC),  9C). 

Theorem  III. 30.  Suppose  that  Assumptions  III. 2,  III. 3,  III.  15,  and  III. 28  are  sat¬ 
isfied,  ( GTPC ),  6C,  (GTP^M(Nj),  and  9CNM(N)  are  defined  as  in  (III. 20),  (III. 52), 
(III.  151),  and  (III.  153),  respectively.  Then  the  pairs  ((GTP^M,Nj),  9CNM(^N^),  in 
the  sequence  {((GTP))M^),  at  are  consistent  approximations  for  the  pair 

{(CTP').O'). 

Proof.  The  proof  that  the  problems  (GTP^M(at))  epi-converge  to  (GTPC)  is  the 
same  as  the  proof  that  the  problems  (GTP/vM(Af))  epi-converge  to  ( GTPd )  given  in 
Theorem  III. 29  above. 

From  the  proof  of  Theorem  III. 29  above,  we  know  that  V/tvmcjv)  (On)  — 
V f(r/),  as  N  — >  oo.  Then,  following  the  same  arguments  as  in  the  proof  of  Theorem 
4.3.6  in  Polak  (1997)  we  see  that  given  any  infinite  sequence  such  that 

r/N  G  HC)JV  for  all  N  G  A f,  which  converges  to  an  rj  G  Hc,  9cNM(N)(on)  — dc(rf),  as 
IV  — ^  oo.  □ 

C.  INDEPENDENT  TARGETS 

We  now  consider  the  case  of  independent  targets,  where  we  proceed  in  a  man¬ 
ner  similar  to  that  used  in  Section  III.B  for  the  case  of  dependent  targets.  While  our 
assumptions  and  approach  are  similar  to  those  used  in  Section  III.B,  the  “informa¬ 
tion  state”  for  the  case  of  independent  targets  is  different  than  it  was  for  the  case  of 
dependent  targets.  The  difference  in  the  “information  state”  is  explained  in  detail  in 
Section  III.C.l. 

This  section  provides  discretization  schemes  that  lead  to  implementable  algo¬ 
rithms  for  the  problems  (FTP6),  (PTPc,e),  ( ITPP ),  and  (FTPC,P),  which  were  defined 
in  Chapter  II.  The  definitions  of  these  problems  in  Chapter  II  were  incomplete  be- 
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cause  they  did  not  include  definitions  for  the  spaces  of  allowable  controls.  This  sec¬ 
tion  begins  by  defining  an  “information  state”  which  we  use  in  conjunction  with  the 
spaces  from  Section  111. A  to  complete  the  definitions  of  ( ITPe ),  ( ITPc,e ),  ( ITPP ), 
and  ( ITPC’P ).  Next,  we  state  our  assumptions  and  define  optimality  conditions  for  the 
generalized  optimal  control  problems.  Then,  we  develop  consistent  approximations 
for  the  time-discretized  search  problems.  Finally,  we  show  that  the  time-  and  space- 
discretized  search  problems  are  consistent  approximations  for  the  original,  continuous 
time-and-spaee  search  problems. 

1.  Information  State  and  Optimal  Control  Problems 

Under  the  assumption  that  the  random  variables  that  the  target  motion  is 
conditioned  upon  are  independent  across  targets,  we  use  (II. 9)  to  define  the  function 
fl  :  H  — »  M  for  any  rj  G  H  by 


K 


fl(v )  = 


exp 


'  al&A 


—  /  rk,l(xri’k(t),yl(t]  al))dt  <fi\al)dal.  (III. 156) 


k=\ 


As  before,  the  superscript  l  is  used  to  denote  the  Ith  attacker  and  l  =  1,  2, ...,  L.  Again, 
to  simplify  the  notation  in  (III.  156)  and  facilitate  the  development  that  follows,  we 
find  it  useful  to  define  a  parametric  “information  state”  denoted  by  zv,l(t ;  a1).  For  any 
a1  G  A,  l  =  1,2,  ...,L,  t  G  [0,1],  and  set  of  searcher  trajectories,  z1,,l(t;al )  represents 
the  cumulative  detection  rate  given  those  searcher  trajectories  and  a1,  and  is  given 
by 

rt  K 

zv,l(t]  a1)  =  /  Vry  (xv’k(s),yl(s-,al))ds,  (III.157) 

Jo  k= i 

or  equivalently  by  the  differential  equation 

I< 

zv,l(s]  a1)  =  J2  rk'1  {xv’k{s),yl(s-,a1))  V  s  G  [0 ,t],  (III.158) 

k=\ 


with  zv'l(0]al)  =  0.  It  is  important  to  note  that  the  “information  state”  in  (III. 157) 
differs  from  the  “information  state”  in  (III. 12)  for  the  dependent  target  case.  In  the 
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dependent  target  case,  the  “information  state”  represented  the  cumulative  detection 
rate  for  all  of  the  searchers  looking  for  all  of  the  targets.  For  the  independent  target 
case,  the  “information  state”  represents  the  cumulative  detection  rate  for  all  of  the 
searchers  looking  for  the  Ith  target.  Using  this  notation,  for  any  rj  G  H.  (III.  156) 
simplifies  to 

fl{rj)  =  f  exp  (— zv,l(l;  a1))  4>l{al)dal.  (III. 159) 

JaleA 

Again,  it  is  useful  to  simplify  the  notation  in  (III.  159)  even  further.  To  this 
end,  for  any  a1  G  A,  l  —  1,  2, L ,  we  also  define  the  function  a1)  :  H  — >  M  by 


f\mal)^F^,x^(t;al)  J,  (III. 160) 

where  F(-;  •)  is  defined  as  in  (III. 15),  and  x,hl(t]  a1)  is  an  augmented  state  defined  by 

x^i^t) 

. zv,l(t ;  a1) 

Using  this  notation,  for  any  r)  G  H.  (III. 159)  simplifies  to 

(III. 162) 


xv\t;al)  = 


G  RnI<+1. 


(III. 161) 


fl(v)=  I 

J  al£A 

To  formulate  the  problem  of  maximizing  the  expected  number  of  targets  de¬ 
tected  during  [0,1],  we  define  the  objective  function  ij)e  :  H  — )•  M  for  any  77  G  H 
by 

L 

r(ri)  =  Y1  f'M-  (III.163) 

1=1 

Then,  we  consider  the  problem 


(. ITPe )  min  '0e('?/). 
?7eH° 


We  also  consider  the  problem 


(III. 164) 


(. ITPc’e )  min  ^e(r/),  (III.165) 

which  allows  for  constraints  on  the  control  input.  It  should  be  noted  that  the  prob¬ 
lems  ( ITPe )  and  ( ITPc,e )  are  both  minimization  problems  despite  the  fact  that  they 
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correspond  to  maximizing  the  expected  number  of  targets  detected.  This  is  because 
the  probability  that  at  least  one  of  the  searchers  detects  the  Ith  target  during  [0, 1]  is 
given  by  1  —  and  hence  the  expected  number  of  targets  detected  during  [0, 1] 

is  given  by  J2i=i  t1  “  fl(v)]  ■ 

To  formulate  the  problem  of  minimizing  the  probability  that  all  of  the  serachers 
fail  to  detect  any  of  the  targets  during  [0,1],  we  define  the  objective  function  : 
H  — >  R.  for  any  rj  G  H  by 

L 

r(v)  =  Y[f‘M.  (hi. 166) 

i=i 

Then,  we  consider  the  problems 


(. ITPP )  min  ijjp(r]),  (III.  167) 

?yeH° 

and 

(ITPC,P)  min  ^ p(r: /).  (III.168) 

i)£Hf 


2.  Optimality  Conditions 

In  this  section,  we  state  our  assumptions  and  give  optimality  conditions  for 
(JTPe),  ( ITPc,e ),  ( ITPP ),  and  ( ITPC,P ).  We  begin  by  deriving  parameterized  differ¬ 
ential  equations  of  the  augmented  dynamics  in  terms  of  the  augmented  state,  xl(t ;  a1), 
defined  in  (III. 161).  For  all  l  —  1,2,  ...,L  and  t  G  [0, 1]  we  define 


hl(x(t),u(t );  a1)  = 


(  h1  {x1  (t) ,  u1  (t))  ^ 


hK ( xR  (t),  uK(t)) 
VEf=i  rk\xk{t\yl{t-al))J 


e  Mn^+1. 


(III. 169) 


For  a  given  a1  £  A,  l  =  1,2 we  write  the  following  parameterized  differential 
equation  to  describe  the  augmented  dynamics 


x\t)  =  hl(x(t),  u(t)\ a1),  t  G  [0, 1],  xl(0)  =  £.  (III. 170) 
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We  let  x 71,1  (■)  denote  the  solution  of  (III. 170)  when  the  input  is  r/  =  (£,u)y  and 
f  =  (£T,  Of.  We  next  state  a  series  of  assumptions  similar  to  those  given  in  Section 
III.B.2,  beginning  with  those  related  to  (j)1  (•)  and  ?/(•;  •). 

Assumption  III. 31.  We  assume  that  <j>1  (•),  l  —  1,2,  ...,L  is  four  times  continuously 
differentiable.  □ 

Assumption  III. 32.  We  assume  that  Assumption  III. 2  holds,  with  a  replaced  by  a1, 
for  l  —  1,  2, L.  □ 


The  next  assumption  is  related  to  rk,l(-,~)  and  /r(-,-;-),  where  we  will  adopt  the 
notation 

/  +  \  n,(+\ \T  \ 


hx(x(t),u(t);a)  = 


hl(x(t),u(t))T 

h«(x(t),u(t))T 


\Lk= i  ( xk  (t) ,  vl  (t ;  «z)  f  / 

where  hlx(x(t),u(t);  a1)  is  a  (nK  +  1)  x  n  matrix  and 


/ 


hlu(x(t),u(ty,a 


l\  A 


^  (x(t),u(t))' 

h«(x(t),u(t))T 

0 


\ 


(III. 171) 


(III. 172) 


where  hlu(x(t),u(t)',  a1)  is  a  ( nK  +  1)  x  m  matrix. 

Assumption  III. 33.  We  assume  that  Assumption  III. 3  holds  with  /?.(•,  •;  a),  hx(-,  •;  a), 
and  huf,  •;  a)  replaced  by  hl  (•,  •;  a1),  hlx(-,  ■]  a1) ,  and  hlu(-,-',al),  respectively.  □ 


We  next  show  that  ifef)  and  f)p  ( • )  are  Gateaux  differentiable  on  H°. 

Proposition  III. 34.  Suppose  that  Assumptions  III.31,  III. 32,  and  III. 33  are  satis¬ 
fied.  Then,  for  any  q  e  H°  and  Srj  G  Tfoog 

(a)  f  (•)  das  a  Gateaux  differential  7;  Srj)  at  rj  given  by 


h2  ’ 


Dfiefq-5q)  =  (Vf  (v),$v) 


(III. 173) 


where  the  gradient  Vfie(r])  is  given  by 


Vrmt)  =  E  (  I  V  al)(t)<f>\al)dal)  ,  Vt  G  [0, 1],  (III.174) 

7^  V-Za^A  / 


and 


(b)  ipp(-)  has  a  Gateaux  differential  Dfi>p(r]]  5rj)  at  rj  given  by 

Dfip(v,  &v)  =  (^V(v),  $v)h2  » 

where  the  gradient  Vfip(r])  is  given  by 


(III. 175) 


E 

i=i 


leA 


Vjl(rj-  al)(t)4>\al)dal 


n 

j=i\j^i 


cPeA 


/J’ (77;  a^cjfi (ar^dcP 


Vt  G  [0,1]. 


(III. 176) 


Proof.  Part  (a)  follows  by  the  same  arguments  as  those  used  in  the  proof  of  Propo¬ 
sition  III. 5,  with  /(•;«)  replaced  by  fl(-',al),  and  the  recognition  that  the  Gateaux 
derivative  of  a  sum  is  the  sum  of  the  Gateaux  derivatives.  Part  (b)  follows  in  a  similar 
manner,  with  an  application  of  a  product  rule  for  Gateaux  derivatives.  □ 

As  in  Section  III.B.2,  we  again  observe  that  Proposition  III. 34  also  holds  under 
weaker  assumptions  on  but  because  we  need  Assumption  III. 31  later  we  adopt 
it  here  as  well.  This  issue  regarding  Assumption  III. 31  also  applies  elsewhere  in  this 
chapter.  Our  next  task  is  to  show  that  V'0e(-)  and  V^p(-)  are  Lipschitz  H-continuous 
on  bounded  subsets  of  Hn. 

Lemma  III. 35.  Suppose  that  Assumptions  III. 31,  III. 32,  and  III. 33  are  satisfied, 
then 

(a)  the  gradient  V^e(-)  is  Lipschitz  H-continuous  on  bounded  subsets  of  H°,  and 

(b)  the  gradient  Vfip(-)  is  Lipschitz  H-continuous  on  bounded  subsets  of  H°. 

Proof.  By  Lemma  III. 9,  sums  and  products  of  Lipschitz  H-continuous  functions  are 
Lipschitz  H-continuous  on  bounded  subsets  of  H".  Based  on  Lemma  III. 6  we  know 
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that  the  components  of  (III. 174)  and  (III. 176)  are  all  Lipschitz  H-continuous,  and 
therefore  the  entire  summations  in  (III. 174)  and  (III. 176)  are  Lipschitz  H-continuous. 

□ 

We  define  nonpositive  optimality  functions  9e  :  H°  — >■  M,  0c,e  :  Hc  — >  M, 
9P  :  H°  — >  M,  and  9C,P  :  H(  — >  M  by 

#'(<?)  =  -|livr0)llk.  (m.177) 

6v(il)  =  mill  (V'0e(i/),  i)'  -  i i)Hl  +  )  ||i)'  -  „f„, ,  (III. 178) 

T]  tnc  Zj 

«p(>?)  =  (hi. ire) 

and 

9C’P(V)  4  min  (V#^),  rj  -  rj)H2  +  J  || r(  -  r) ||^2 ,  (III.180) 

T]  tnc  Z 

which  dehne  optimality  conditions  for  ( ITPe ),  (JTPc,e),  ( ITPP ),  and  (ITPC,P),  re¬ 
spectively. 

Proposition  III. 36.  Suppose  that  Assumptions  III.31,  III. 32,  and  III. 33  are  satis¬ 
fied. 

(a)  9e(-),  9c,e(-),  9P(-),  and  9C,P(-)  are  H°-continuous  functions. 

(b)  If  fj  E  H°  is  a  local  minimizer  of  ( ITPe ),  then  9e(rj)  =  0. 

(c)  If  fj  E  H,  is  a  local  minimizer  of  ( ITPc,e ),  then  dc,e(r))  =  0. 

(d)  If  rj  E  H°  is  a  local  minimizer  of  ( ITPP ),  then  9p{fj)  =  0. 

(e)  If  fj  E  Hc  is  a  local  minimizer  of  ( ITPC,P ),  then  9c,p('fj )  =  0. 

Proof.  The  proof  follows  the  same  arguments  as  those  for  the  proof  of  Theorem 
4.2.3  in  Polak  (1997),  with  Proposition  III. 34  taking  the  place  of  Corollary  5.6.9  from 
Polak  (1997)  and  Lemma  III.35  taking  the  place  of  Theorem  4.1.3  from  Polak  (1997). 

□ 
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3.  Consistent  Approximations 

In  this  section  we  define  the  approximating  problems  (. ITP ^),  (ITP^e),  (. ITP 
and  (ITP^p),  and  present  consistency  conditions  for  them.  As  in  the  case  of  de¬ 
pendent  targets,  we  divide  our  development  into  two  subsections.  Both  subsec¬ 
tions  develop  consistent  approximations  for  the  pairs  (( ITPe),9e ),  ((/TPc,e), 6c,e), 
(( ITPP),9P ),  and  ((PTPC,P),  6>c’p),  but  the  first  subsection  only  deals  with  time  dis¬ 
cretization  while  the  second  subsection  considers  time  and  space  discretization. 

a.  Time- Discretized  Problems 

We  again  consider  the  approximate  solution  of  (11.22)  by  means  of  for¬ 
ward  Euler’s  method,  which  was  given  in  (III. 60).  Simultaneously,  we  approximately 
solve  (III.  158)  also  by  forward  Euler’s  method.  For  any  rj  =  (£,u)  G  Hjy,  a1  G  A,  and 
N  G  A f,  we  set  0;  a1)  —  0,  l  —  1,  2, ...,  L,  and  for  any  j  —  0, 1, ...,  N  —  1, 

1  K 

zTi  (U  +  1)/Af;a‘)  -  A  (j/N;al)  =  =  £>*•'  ( x"*(ilN),y\j/N-al ))  .  (III.  181) 

V  k= 1 


Using  the  discretized  “information  state”  given  by  the  recursion  (III.  181), 


we  define  the  approximate  objective  functions  ^ e,N  : 

for  any  G  Hjy  and  IV  G  A/”  by 

Hat  — >  M  and  ij}p,N 

\  — y  M 

^h(v)  -  !  exp  at 

i=i  V 

4>l(al)dal, 

(III. 182) 

and 

L 

Vn(v)  —  IJ  /  exp  (~zn(^  al ) 

i=i  J^A  V 

4>l(al)dal. 

(III. 183) 

We  also  define  the  individual  components  of  (III. 182)  and  (III. 183)  by 

fN(v)  =  J  ;  ^exp(-4i(l;ai)j(/)Z(ai)d«Z,V/  =  l,2,...,L.  (III.184) 

Again,  for  the  sake  of  notational  simplification,  for  any  a1  G  A,  l  =  1,  2, ...,  L,  we  also 
define  the  functions  a1)  :  D°N  — >■  M  by 

PN(v,al)^F{i,xf(t-,a1)),  (III.185) 
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where  F  is  as  defined  in  (III.  15)  and  xvJ'(j /N]  a1)  is  an  augmented  state  defined  by 


^(j/N-a1)  = 


xUj/N) 


L,  j  =  0, 1, N  —  1. 


W  /  >  /  I  /  ,  I  l  ill 

\zn(J/n ; « )/ 

Hence,  for  any  IV  G  A/",  we  define  the  following  approximating  problems 


(III. 186) 


min 

^n(v), 

(III. 187) 

(/rpy) 

min 

jjeHc,jv 

(III. 188) 

(/rr-j) 

min 

ueH®, 

Vn(v), 

(III. 189) 

(ITPtf) 

min 

rM- 

(III. 190) 

We  next  consider  the  differentiability  of  fieN{-)  and 

Proposition  III. 37.  Suppose  that  Assumptions  III.31,  III. 32,  and  III. 33  are  satis¬ 
fied,  and  N  G  A I.  Then,  for  any  rj  G  and  Sr /  G  h/00,2 

(a)  has  a  Gateaux  differential  Srf)  =  (W/tp%(r}),Srj) H2,  where 

VrN(v)(t)  =  E  f  VvflN(W,al)(t)fi\al)dalyt  G  [0,1],  (III.191) 

1=1  JaleA 


(b)  has  a  Gateaux  differential  Di/fN{r]\  Srj)  =  (V'0^-(^),  dfi) Hn,  where 

=  (  /  ^nflN(v,^l)(t)fi\al)dal  JJ  /  /^(r?;  (aJ)daJ 

1=1  \UaleA  J 


Vt  G  [0,1]. 


(III. 192) 
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Proof.  Part  (a)  can  be  proven  by  the  same  arguments  used  in  the  proof  of  Propo¬ 
sition  III. 5,  with  /(•;«)  replaced  by  flN(-]al),  and  the  recognition  that  the  Gateaux 
derivative  of  a  sum  is  the  sum  of  the  Gateaux  derivatives.  Part  (b)  follows  in  a  similar 
manner,  with  an  application  of  a  product  rule  for  Gateaux  derivatives.  □ 

Next  we  show  that  Vfi%(-)  and  V^(-)  are  Lipschitz  H ^-continuous 
on  bounded  subsets  of  H°N. 

Lemma  III. 38.  Suppose  that  Assumptions  III. 31,  III. 32,  and  III. 33  are  satisfied, 
then 

(a)  the  gradient  Wfi>eN(-)  is  Lipschitz  Hy  continuous  on  bounded  subsets  of  H°N,  and 

(b)  the  gradient  VV’jv(-)  is  Lipschitz  Phy-continuous  on  bounded  subsets  of  Hy 

Proof.  By  Lemma  III.9,  sums  and  products  of  Lipschitz  H ^-continuous  functions  are 
Lipschitz  PPy-continuous  on  bounded  subsets  of  Hy  Based  on  Lemma  111.10(b)  we 
know  that  the  components  of  (III. 191)  and  (III.  192)  are  all  Lipschitz  Hycontinuous, 
and  therefore  the  entire  summations  in  (III.  191)  and  (III. 192)  are  Lipschitz  Hy 


continuous.  □ 

For  any  N  e  A f,  we  define  nonpositive  optimality  functions  9%  :  H°N  — > 
R,  6%e  :  HcJV  — ►  R,  6PN  :  U°N  ->■  R,  and  6%p  :  HciV  ->  R  by 

=  -i||V*0)llk.  (HI-193) 

««'(>))-  min  v'-  <))h2  +  i  W  ~ ’1\\h,  .  (III.194) 

T)  Grlc,AT  ^ 

«£t(!>)  =  -i||V*(>))llk.  (HI-195) 

and 

°nP  (v)  =  nhn  {VfipN(r)),  r(  -  r ])  H2  +  \  ||?/  -  vfH2 ,  (III.196) 

T)  Grlc,JV  ^ 


which  characterize  stationary  points  of  ( ITP ^),  (ITPfifi),  ( ITP y),  and  (IT Pfip), 
respectively. 
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Proposition  III. 39.  Suppose  that  Assumptions  III. 31,  III. 32,  and  III. 33  are  satis¬ 
fied. 

(a)  9eN( •),  9<ff(-) ,  0PN{-),  and  9cfif’(-)  are  H^-continuous  functions. 

(b)  If  fj  G  ti%  is  a  local  minimizer  of  ( ITP then  On(v)  =  0- 

(c)  If  77  G  HCijv  is  a  local  minimizer  of  (ITPffi),  then  9ff(rj)  =  0. 

(d)  If  rj  G  is  a  local  minimizer  of  (ITPfi),  then  9pN(fi)  =  0. 

(e)  If  f)  G  HCijv  is  a  local  minimizer  of  (ITPff),  then  9cffi(rj)  =  0. 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Proposition  1.1.6  in 
Polak  (1997),  with  the  norms  and  inner  products  replaced  with  their  H2  equivalents. 

□ 

As  is  the  case  in  Section  III. B. 3a  with  epi- convergence  of  ( GTP^ )  to 
( GTP ),  it  is  not  possible  to  establish  epi-convergence  of  ( ITP %)  to  ( ITPS )  or  (IT  Pfi) 
to  (ITPP).  We  define  the  problems 


(ITPS 

min 

^ e(v ), 

(III. 197) 

-)£H  0 

(ITPS 

min 

V(v), 

(III. 198) 

because  it  is  possible  to  establish  epi-convergence  of  ( ITP '(f)  to  (IT  Pffi)  and  ( ITP 'jf) 
to  (ITPff).  We  will  use  the  problems  (IT  Pf)  and  (ITPp),  with  the  following  assump¬ 
tion. 

Assumption  III. 40.  We  assume  that  all  local  and  global  minimizers  of  (IT  Pf)  and 
(ITPffi)  are  in  H°.  □ 

Again,  in  a  manner  similar  to  that  of  Section  3.3  of  Polak  (1997)  we 
next  show  that  the  pairs  ((IT P(f) ,  0eN)  in  the  sequence  {((/TP^),  0£n)}n&n  and  the 
pairs  ((ITPfi),  0PN)  in  the  sequence  {(( ITP '(f),  9pn)}n^m  are  consistent  approximations 
for  the  pairs  ((ITPf),  6e)  and  ((ITPffi,  9P),  respectively.  In  order  to  establish  epi- 
convergence,  we  will  need  the  following  intermediate  result. 


74 


Lemma  III. 41.  Suppose  that  /3l  :  H  — y  [0,1]  and  6lN  :  — *  [0,1],  l  =  1,2,...,  L, 

iVeN,  are  such  that  for  all  r)  G  H,  l  —  1,  2, ...,  L,  and  N  G  N, 

|/4M  -/^)|  <  A(IV)  (III. 199) 

where  A  :  N  — >  [0,  oo)  is  a  strictly  decreasing  function  with  A (N)  — >  0,  as  N  — >  oo. 
TAen  for  all  N  G  N  and  g  G  H, 

(a) 

L  L 

f3l(g)\<  LA(N)  (III. 200) 

«=i  ;=i 

and 

(b)  there  is  a  constant  AA  <  oo  such  that 

L  L 

1H  ’  •  II  /3l(ri)\<KvA{N).  (III. 201) 

i=i  i=i 

Proof.  To  prove  part  (a),  we  expand  the  summations  in  (III. 200),  combine  terms, 
and  use  the  triangle  inequality  to  write 

L  L  L 

EA(i  )-£/%)  = 

i=i  i=i  i=i 

L 

1=1 

<  LA(N)  (III. 202) 

For  the  proof  of  part  (b),  we  write  out  both  the  products  in  (III. 201) 
and  express  f3lN(g)  in  terms  of  j3l(  g)  and  A  (A),  then  because  (3\g)  G  [0,1]  and 
/ 3lN(g )  G  [0, 1]  for  all  l  we  have  that 

L  L 

1=1  1=1 


< 


< 


n  (/8'm + Aw)  -n  PM 

i=i  i=i 

E(  f)G(iv)]'. 


(111. 203) 

(111. 204) 
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where  the  final  inequality  follows  from  taking  the  product  of  the  (/ 6l(r /)  +  A  (TV))  in 
(111.203)  and  combining  like  terms.  Then  because  A (TV)  — y  0  as  TV  — y  oo,  there  exists 
TV  €  N  such  that  for  all  TV  >  TV  there  is  a  constant  Kn  <  oo  such  that 

E  (f)  [AM]'  =  “W  +  E  (t)  [AW]'  <  A'.A(iV),  (III. 205) 

1=1  A  '  Z  =  2  A  / 

□ 

We  now  show  the  epi- convergence  of  (IT  Pf)  to  (IT  Pf)  and  ( ITP '^) 

to  (FTPS). 

Theorem  III. 42.  Suppose  that  Assumptions  III. 31,  III. 32,  III. 33,  and  III. 40  are 
satisfied.  Then 

(a)  (IT  Pf)  epi-converges  to  (ITPff),  as  TV  — »  oo,  and 

(b)  (IT  Pf)  epi-converges  to  ( ITP( ff,  as  TV  — y  oo. 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Theorem  III. 17.  We 
then  invoke  Lemma  111.41(a)  and  (b)  to  complete  the  proofs  of  parts  (a)  and  (b), 
respectively. 


□ 

In  order  to  show  consistency  of  approximation  for  the  pairs  ((ITPf),  9eN), 
(fITPff),  Off),  (( ITPf ),  0PN),  and  (( ITPff ),  Off)  we  will  need  the  following  two 
results. 

Lemma  III. 43.  Suppose  that  there  exist  constants,  Cl  <  oo  and  ClN  <  oo,  and  that 
the  functions  V /3l  :  H°  — *  H.  V f3lN  :  H^r  — >  H^v,  (3l  :  H  — )■  [0, 1],  and  (3lN  :  — » 

[0, 1],  l  =  1,  2, ...,  L,  TV  E  N,  satisfy 

||V/^)|| h2  <  C\  WI  =  1,2,...,L,VeU°  (HI-206) 

II VPM\\h2  <  C1n,\/1  =  1,2,...,L,veU%,NeN  (III.207) 

\Pn(v)~P1(v)\  <  A(TV),  V(  =  1)21..,L,^HW)JV6N  (HI-208) 

and 

\\Vf3lN(V)  -  V/3l(r,)\\H2  <  A  (TV),  V/  =  1,2  ,...,L,rj  E  H  °n,Ne  N,  (HI.209) 

where  A  :  N  — *  [0,  oo)  is  a  strictly  decreasing  function  with  A  (TV)  — »  0,  as  TV  — *  oo. 
Then, 
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(a)  for  all  IVeN  and  rj  G 


L 


L 


Ev&M-Evrt?) 


<  LV2A(N), 

h2 


and 

(b)  there  is  a  constant  KnQ  <  oo  such  that  for  all  N  G  N  and  rj  G 


(III. 210) 


evam  n  n 


«=i 


j=i|jyy 


/=i 


i=lbW 


<  KvGA(N). 


h2 


(III. 211) 


Proof.  We  begin  by  proving  part  (a).  We  know  that 


EvAm-EvA>7) 


/=! 


1=1 


L 

<EIIvAo)-v/j,o)||H3. 

#2  i=1 


(III. 212) 


Let  l  —  1,2,  ...,L,  IV  G  N,  and  rj  G  be  arbitrary,  then  based  on  the  definition  of 


the  H2  norm  1 1 V/3jv (-^7)  —  V /3l  (rj)\\ <  A (IV)  implies  that 

||vA(r,)-V/3'(,)||2Hi  =  ||V4A(r,)  -  V^'(,)||2 

+  /  ||v„AWW-v„/)i0)(i)||2<ii 

do 

<  [A(IV)]2  +  [A(IV)]2 ,  (III. 213) 


where  SlN(r /)  G  Mn,  V^/3l(r])  G  Mn,  Vu(3lN(r])  G  U,  and  Vu/3l(r))  G  U.  Hence 

||V^(77)  —  V/3i(77)||jff2  <  V^A(IV),VI  =  1,2,...,L,  (III.214) 

which  completes  the  proof  of  part  (a). 

For  the  proof  of  part  (b),  let  glN{rj)  A  Pn(v)  and  gl(rj)  A 

nUwFti)- Then’ 

L  L  L  L 

evaw  n  aw-ew’w  n 

i  =  1  l=lbW  i  =  1  rr 


77 


=  E  -  Y  s'wvp'w 

1=1  1=1  h2 

L 

<  Yb'MvM-s'Wvp'WW^.  (in-215) 

(=1 

Based  on  the  definition  of  the  Hi  norm  we  have 

IIAMVAO)  -s'Mv/J'MIIb,  = 

||9t(>f)V£A0)  -  s'WV^'WII2  +  f  II g’MvAm)  - a’(vW(ri)(t)\\2dt 

Jo 

(III. 216) 

Based  on  Lemma  111.41(b),  there  exist  constants  Kl  <  oo,  l  =  1,2,  ...,L,  such  that 
V7V  G  N,  and  Vr?  e 

WnM  -  g‘(ri)\  <  K‘A(N).  (III.217) 

We  then  consider  the  first  term  on  the  right-hand  side  of  (III. 216),  which  can  be 

rewritten 

IIAWVjAW-s'l^v^'Wlf 
=  IkMvAM  -  a‘Mvsp‘ („)  +  (g‘„(v)  -  g'M)  vf&(,)||2 

<  s'lnf  HwAfa)  -  v£/3‘(,,)||2  + ||(A0)  -  g'w) 

<  |A(Af)]2  +  (sUv)  -  g‘(’l))2 1| v£AO)||2 

<  [A(AQ]2  +  (Kg)2  |A(/V)|2  ||V£A(r,)||2 

<  [A(iV)]2  +  (ClN)2  (Kg)2  [AfA7")]2 

<  (l  +  (C-i-)2  (A'J)2)  |A(7V)]2 .  (III.218) 
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Next,  we  consider  the  second  term  in  the  sum  on  the  right-hand  side 
of  (III. 216),  which  can  be  rewritten 

J  0 

=  [  II 9l(v)VuPlN(v)(t)  -  9l{ri)VuP\ri){t)  +  (glN(rj)  -  gl{r)))  V u/3lN(v)(t)\\2  dt 

Jo 

<  s'(v)2  f  ||v„AW(()  — v„/3‘(r,)(t)||2di+  /  IKAW-s'W)  vAinmfdt 

Jo  Jo 

<  [A(JV)]2+  [\g‘M-g‘M)2\\vA(vm\\2dt 

Jo 

<  [A(/V)]2+(A'J)2[A(Af)]2  f'\\VuP‘M(t)\\2dt 

Jo 

<  [A(JV)]2+((C',)2(Aj)2)[A(Af)]2<  (l  +  (C‘,)2(Aj)2)[A(Af)]2.  (III.219) 

Hence 

IIAMvAM  -  a'MVfiHri) ||„2  <^2(1  +  ( c‘N )2  (Jr')2) a(jv), Vi  =  1, 2, l. 

(III. 220) 

Then  Knc  =  22t=i  \J 2  ^1  +  (Cat)"  (A^J,  and  the  proof  is  complete.  □ 

Proposition  III. 44.  Suppose  that  Assumptions  III.31,  III. 32,  and  III. 33  are  satis¬ 
fied,  then  for  every  bounded  subset  S  C  H°, 

(a)  there  exists  a  constant  K^p  <  oo  such  that,  for  any  N  e  M  and  rj  G  S  ft  HN 

IIV^W-v^mIIh,  <  ^  (111221) 

and 

(b)  there  exists  a  constant  Knp  <  00  such  that,  for  any  N  G  J\f  and  77  £  S  fl 

l|V*(>))  -  V^(ij)|k  <  (111.222) 

Proof.  From  Proposition  III. 16,  we  deduce  that 

[  Pn(V’,<*1)  -  fl(v,al)  (j)\al)dal  <  (III.223) 

da'eu  iV 
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The  proof  then  follows  the  same  arguments  as  the  proof  of  Proposition  III.  18  with 
Propositions  III. 5  and  III. 11  replaced  by  Propositions  III. 34  and  III. 37,  respectively. 
We  then  invoke  Lemma  111.43(a)  and  (b)  to  complete  the  proofs  of  parts  (a)  and  (b), 
respectively.  □ 

We  now  show  that  the  pairs  (( ITP %)  !  Gif)  in  the  sequence 
{{{ITPff),6eN)} NeM  and  the  pairs  ((ITPf),9pN)  in  the  sequence  {((JTP£),  9pN)}NeM 
are  consistent  approximations  for  the  pairs  ((ITP(f),9e)  and  {(IT Pff) ,  9P) ,  respec¬ 
tively. 

Theorem  III. 45.  Suppose  that  Assumptions  III. 31,  III. 32,  III. 33,  and  III. 40  are 
satisfied,  ( ITP %),  ( ITP p),  9e,  9P,  (ITPf),  ( ITP '((),  9eN,  and  9PN  are  defined  as  in 
(III.  197),  (III.  198),  (III.  177),  (III.  179),  (III.  187),  (III.  189),  (III  193),  and  (III.  195), 
respectively.  Then 

(a)  the  pairs  ((ITPf),  9eN ),  in  the  sequence  {((/TP))),  9%)}n£j^  are  consistent  ap¬ 
proximations  for  the  pair  ((/TPcez),  9e),  and 

(b)  the  pairs  ((ITPf),  9PN ),  in  the  sequence  {((/TPj),  9pn)}n&m  are  consistent  ap¬ 
proximations  for  the  pair  ((IT Pp cl) ,  9P) . 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Theorem  III. 20  with 
Theorem  III. 17,  Lemma  III.6,  and  Proposition  III. 18  replaced  by  Theorem  III. 42, 
Lemma  III. 35,  and  Proposition  III. 44,  respectively.  □ 

We  next  show  that  the  pairs  ((ITPfe)  ,0(f)  in  the  sequence 
{((ITPc/),9(f)}NeM  and  the  pairs  ((ITPfp),  9(f)  in  the  sequence  {((ITPfp),  9(f)} NgJ^ 
are  consistent  approximations  for  the  pairs  ((/TPc,e),  9c,e)  and  ((ITPC,P),  9C,P ),  respec¬ 
tively. 

Theorem  III. 46.  Suppose  that  Assumptions  III. 31,  III. 32,  III. 33,  and  III. 40  are  sat¬ 
isfied,  ( ITPc’e ),  (ITPC,P),  9c’e,  9C’P,  (ITPfe),  (ITPfif),  9(f,  and  9(f  are  defined  as  m 
(III.  165),  (III.  168),  (III.  178),  (III.  180),  (III.  188),  (III.  190),  (III.  194), and  (HI- 196), 
respectively.  Then 

(a)  the  pairs  ((ITP'fif),  9(f),  in  the  sequence  {((ITPfe),  9(f)}]y£M  are  consistent 
approximations  for  the  pair  ((/TPc,e),  9c,e),  and 

(b)  the  pairs  ((ITPfp),  0(f  ),  in  the  sequence  {((ITPff),9(f)}N^M  are  consistent 
approximations  for  the  pair  ((ITPC,P),9C’P). 
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Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Theorem  III. 21  with 
Theorems  III.  17  and  III. 20  replaced  by  Theorems  III. 42  and  III. 45,  respectively.  □ 


b.  Time-  and  Space- Discretized  Problems 

We  next  consider  the  time-  and  space-discretized  problem.  We  make 
use  of  the  integration  rule  Im  given  in  (III.  102)  to  define  the  approximate  objective 
functions  — »  M  and  fiPNM  :  — >  M  for  any  rj  G  Hat,  N  G  Af ,  and 

M  G  N  x  N  by 


^nm(v)  ~  Im  (exp  ~zn\1\  •)  ^(0)  » 


and 


i=i 

L 


^ nm(v )  -  n  Im  (exp  ~zn\^ 


1=1 


(III. 224) 


(III. 225) 


We  next  consider  the  differentiability  of  and  ^ nm (')• 

Proposition  III. 47.  Suppose  that  Assumptions  III.31,  III. 32,  and  III. 33  are  satis¬ 
fied,  N  G  Af,  and  M  6  N  x  N,  then  for  any  q  G  H°N  and  Sr)  G 

(a)  iPnm(')  Pas  a  Gateaux  differential  DfieNMfq]Sq)  =  Sq)Hn,  where 


=  E  b"  KA(d;-)W^(-)1)  ,vt  e  [0,1],  (hi. 226) 


z=i 


and 


(b)  V’jvm(')  has  a  Gateaux  differential  D'il/fiM{r)\  Srf)  =  fS'ifipNM{ri)i  ^v)H  >  where 


z=i 


L 

/m[v,/^;-)(^(-)]] 

n  iM 

j= 

Vt  G  [0,1]. 


(III. 227) 


Proof.  The  proof  of  part  (a)  follows  the  same  arguments  used  in  the  proof  of  Propo¬ 
sition  III. 22,  with  /(•;«)  replaced  by  /] y(-;cd),  and  with  the  recognition  that  the 
Gateaux  derivative  of  a  sum  is  the  sum  of  the  Gateaux  derivatives.  Part  (b)  follows 
in  a  similar  manner,  with  an  application  of  a  product  rule  for  Gateaux  derivatives.  □ 
Next,  we  show  that  and  V^M(-)  are  Lipschitz  Hyv-continuous 

on  bounded  subsets  of  H°N. 
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Lemma  III. 48.  Suppose  that  Assumptions  III. 31,  III. 32,  and  III. 33  are  satisfied, 
then 

(a)  the  gradient  is  Lipschitz  H ^-continuous  on  bounded  subsets  of  H°N, 

and 

(b)  the  gradient  V^M(?/)  is  Lipschitz  H/v-contirmous  on  bounded  subsets  of  H°N. 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Lemma  III. 38,  with 
integration  replaced  by  Im-  □ 

For  the  reasons  discussed  in  Section  III. B. 3b,  in  order  to  conduct  the 
analysis  that  follows  we  again  make  the  assumptions  delineated  in  Assumption  III. 24. 
Based  on  the  choice  of  Composite  Simpson’s  rule  as  the  numerical  integration  scheme, 
we  find  it  necessary  to  show  that  the  partial  derivatives  of  flN(ip,  ■ )fil( •)  and 
Vrj./'.v (h;  ')^(')  UP  to  and  including  the  fourth-order  are  bounded  for  any  choice  of 
rj  G  H%,  any  a1  G  A,  and  any  N  G  A f.  The  bounding  of  the  partial  derivatives 
of  flN(ip,  ■ )4>l{ •)  and  VT,/^(r/;  -)0Z(-)  is  necessary  in  order  to  prove  the  convergence  of 
^nm(v)  to  ^(v),  eNM(v )  to  Vr(v),  Vnm (v)  to  ^p(yy),  and  V^M(r/)  to  Vfip(r]). 

To  facilitate  these  proofs  we  begin  by  defining  some  notation.  For  any  rj  G 
a1  G  A,  l  =  1,  2, ...,  L,  N  G  A/”,  and  j  —  0, 1, ...,  N  —■  1,  we  define 


C1K)  ±  exp 


(III. 228) 


(III. 229) 


(III. 230) 

and 

CS(«‘)  =  Pn  <*')  ■  (III. 231) 

We  again  note  that  (){(•),  Cs(')>  an(l  CK')*  ^  =  1,2, ... ,L ,  depend  on  rj  and  N. 
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We  next  show  that  the  partial  derivatives  of  ([  up  to  and  including  the 

fourth-order  are  continuous  and  bounded  for  any  choice  of  rj  G  a1  G  A,  l  = 
1,  2, L,  and  N  G  A f. 

Lemma  III. 49.  Suppose  that  Assumptions  III. 31,  III. 32,  and  III. 33  are  satisfied  and 
S  is  a  bounded  subset  ofH%.  Then, 

(i)  C'(-)eC4(i)  Vi  =  1,2,  3, 4, 
and 

(ii)  there  exists  C  <  oo,  such  that  for  all  rj  G  S,  j  —  0,1,  ...,N  —  1,  a1  G  A, 
l  —  1,2, L,  and  N  G  M 


ddr 


<C  Vi 


1,2 ,V/t  =  1,  2, 3, 4,  Vk  =  1,2,  3, 4. 


(III. 232) 


Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Lemma  III. 25  with 
Assumptions  III.  1,  III. 2,  and  III. 3  replaced  by  Assumptions  III. 31,  III. 32,  and  III. 33, 
respectively.  □ 

In  order  to  prove  epi-convergence  and  consistency  of  approximation,  we 
will  need  the  following  proposition. 

Proposition  III. 50.  Suppose  that  Assumptions  III. 24,  III. 31,  III. 32,  and  III. 33  are 
satisfied.  Then  for  every  bounded  subset  S  C  H°,  there  exist  constants  Kes  <  oo, 
KttS  <  oo,  and  Knj\ ,  AV-ra  <  oo  such  that,  for  any  N  G  A f,  for  any  MgN  3  x  N34, 
and  rj  G  S  fl  HN, 

(a)  (i)  | ^(3)  ~  4>nm(v) I  <  ^  +  (jgrfjr , 

(ii)  || V^e(^)  - 'WW^)lk  <  +  ^frij4  +  pifrij4, 

and 

<b)  <i)  iv>p(ij)  -  V'wmWI  <  V2  +  fjIV1  + 

(ii)  ||V^(,,)  -  V«„( r;)||Hj  <  Y  +  <JfSy>  +  (jg^> 

and  Kef  and  Knp  are  defined  as  in  Proposition  III.  44- 


4Recall  that  N3  =  {m  £  2N  +  l|m  >  3},  as  defined  previously  in  Proposition  III. 26. 
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Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Proposition  III. 26  with 
Proposition  III. 18,  Lemma  III. 25,  and  Ci('),  •  ••,  C4(')  replaced  by  Proposition  III. 44, 
Lemma  III. 49,  and  (){(•),  respectively.  □ 

For  any  N  e  AT,  and  MeNxN,  we  define  the  following  approximating 

problems 


(ITPeNM) 

min 

(III. 233) 

(upyM) 

min 

i)6HCiW 

(III. 234) 

( ITP%m ) 

min 

'll)PNM{ri)i 

(III. 235) 

and 

(ITPc/m) 

min 

t)6HCiW 

(III. 236) 

For  any  N  e  AT,  and  M  e  N  x  N,  we  define  nonpositive  optimality 

functions  deNM  :  M,  9%eM  :  Hc>Ar 

— x  M,  — x  M,  and  9 

NM  :  HCijv  — *  K- 

by 

-\\m 

Nm(v)\\h2i 

(III. 237) 

8Cnm(v)  =  min  (WW^W  -  rj)H2  +  J  \\rf  -  rj\\2H2 ,  (III.238) 

=  -^11  V«„0)IIL  (III. 239) 

and 

=  min  (V^M(v),  v'  ~  v)h2  +  \  II v'  ~  v\\h2  ,  (III.240) 

Pe  HCiiV  2 

which  characterize  stationary  points  of  (IT P^M),  (ITP^eM),  (ITP^M),  and  (ITP^PM), 
respectively. 
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Proposition  III. 51.  Suppose  that  Assumptions  III. 24,  III.32,  and  III. 33  are  satis¬ 
fied. 

(a)  0eNM(-),  0°nm ( ' ) ?  Qnm(')i  and  9nm(')  arc  H^-continuous  functions. 

(b)  If  f)  G  is  a  local  minimizer  of  (ITP^M),  then  9eNM(I))  =  0. 

(c)  If  r)  G  Hc  at  is  a  local  minimizer  of  (ITP^eM),  then  @nm(v)  ~  0- 

(d)  If  77  G  is  a  local  minimizer  of  (ITP^M),  then  0pNM(rj)  =  0. 

(e)  If  r)  G  Hc  at  is  a  local  minimizer  of  (ITP^-PM),  then  ocMv)  =  0. 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Proposition  1.1.6  in 
Polak  (1997),  with  the  norms  and  inner  products  replaced  by  their  H2  equivalents.  □ 
It  is  not  possible  to  establish  epi-convergence  of  (ITP^M,N^)  to  ( ITPe ) 
or  (ITP^M,Nj)  to  ( ITPP ),  but  it  is  possible  to  establish  epi-convergence  of  (ITP^M^) 
to  (IT Pfi)  and  ( ITP^M^N ))  to  (. ITP ^).  We  now  show  that  the  family 
{((ITPnM(N))i  9 nm(N))}n sa/-  is  a  sequence  of  consistent  approximations  for 
((IT P^i) ,  9e) ,  and  the  family  {((ITP^M,N^),  9pnm^)}n£M  is  a  sequence  of  consistent 
approximations  for  (( ITP p),  6P )  by  showing  that  the  conditions  of  Definition  III. 2  are 
satisfied. 

Theorem  III. 52.  Suppose  that  Assumptions  III. 28,  III. 32,  III. 33,  and  III.40  are  sat¬ 
isfied,  (. ITPt 3),  (ITPPcl),  9e,  9p,  ( ITPeNM{N) ),  (ITPpnm{n)),  eeNM(N),  and6pNM{N)  are 
defined  as  in  (III.  197),  (III.  198),  (III.  177),  (III.  179),  (III. 233),  (III. 235),  (III. 237), 
and  (III.  239),  respectively.  Then 

(a)  the  pairs  ((ITP^M{N)),  9eNM{ N)),  in  the  sequence  {((ITP^M{N)),9eNM{N))}NeAf 
are  consistent  approximations  for  the  pair  ((ITP^),9e),  and 

(b)  the  pairs  ((ITPpNM{N)),  9PNM{N)),  in  the  sequence  {((ITPpNM{N)),  9pNM{N))}NeM 
are  consistent  approximations  for  the  pair  ((ITPp(),  Qp). 

Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Theorem  III. 29  with 
Proposition  III. 26  and  Theorems  III.  17  and  III. 20  replaced  by  Proposition  III. 50  and 
Theorems  III. 42  and  III. 45,  respectively.  □ 

We  now  show  that  the  pairs  ((ITP^eM,N^),9Cj^M,N^)  in  the  sequence 
WTPcN‘Mm)  ,  ^7VM(V))}AreA/'  are  consistent  approximations  for  the  pair  ((ITPc,e),  9c,e). 
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We  also  show  that  the  pairs  {{ITPffM,N^),6Cj(fM,N^)  in  the  sequence 

{((ITPc/m{  n))  ,  0'nm(n))}n^  are  consistent  approximations  for  the  pair  ((/TPC,P),  9C,P). 


I II. 33,  and  III.40  are  sat- 
9 


Theorem  III. 53.  Suppose  that  Assumptions  III.  28,  III. I 
isfied,  ( ITP «),  (ITPC,P),  0“  F*,  (1TP"m{n)),  ( ITP‘NM(N)I 

’  UNM(N)> 

are  defined  as  in  (III. 165),  (III.  168),  (III.  178),  (III.  180),  (III.  234),  (HI- 236),  (III.  238), 
and  (III. 240),  respectively.  Then 


and  9C,P 
ana  vNM(N) 


(a)  the  pairs  ((ITPcfeM{N)),  9C£M{N)),  in  the  sequence  {((ITP%eM{N)),  Oc£m{n))}n&m 
are  consistent  approximations  for  the  pair  (( ITPc’e),9c,e ),  and 


(b)  the  pairs  ((ITPC/M{N)),  9CNPM{N)),  in  the  sequence  {((ITPfpM{N)),  ^fM{N))}NeM 
are  consistent  approximations  for  the  pair  (( ITPC,P),9C,P ). 


Proof.  The  proof  follows  the  same  arguments  as  the  proof  of  Theorem  III. 30,  with 
Theorem  III. 29  replaced  by  Theorem  III. 52.  □ 


IV. 


RATE  OF  CONVERGENCE  ANALYSIS 


A.  INTRODUCTION 

In  this  chapter,  we  develop  rate  of  convergence  results  by  expressing  the  rate 
of  convergence  in  terms  of  the  computational  work,  rather  than  the  typical  number  of 
iterations  or  level  of  discretization.  We  relate  the  compuational  work  to  the  number 
of  iterations  as  well  as  to  the  levels  of  discretization  by  making  computational  work 
assumptions.  This  relation  allows  us  to  determine  a  guaranteed  rate  of  convergence 
for  the  error  between  the  objective  function  evaluated  at  iterates  generated  from  the 
discretized  problems  and  the  optimal  value  of  the  original  problem  as  a  function  of  the 
computational  work.  Because  these  results  are  applicable  to  a  range  of  approximation 
problems,  we  prefer  to  develop  our  results  using  abstract  problems  as  we  did  in  Section 
III.B.3.  We  again  define  the  infinite-dimensional  problem 

(P)  min  f(x),  (IV.  1) 

x£X 

where  /(•)  and  X  are  defined  as  in  Section  III. B. 3a.  For  all  N,  M  e  N,  let  / nm  '■ 
Bn  — »  M  be  a  lower  semicontinuous  function  that  approximates  /(•)  on  Bn ,  and  let 
XN  C  Bn  be  an  approximation  to  X,  where  Bn  arid  M  are  defined  as  in  Section 
III. B. 3a.  We  then  define  the  family  of  finite  dimensional  approximating  problems 

(Pnm)  nrin  fNM(x),  N,  M  E  Af.  (IV.2) 

x£Xn 

We  note  that  x  is  now  a  decision  variable,  and  it  no  longer  represents  the  physical  state 
of  the  searcher.  We  also  note  that  the  problems  ( GTP ),  ( GTPC ),  (/TPe),  (JTPc,e), 
( ITPP ),  and  (ITPC,P)  discussed  in  Chapter  III  are  examples  of  (P).  The  problems 
(GTPnm),(GTPcnm),  (ITP^m),  ( ITPc/m ),  ( ITPpNM ),  and  ( ITPC/M )  discussed  in 
Chapter  III  are  examples  of  (Pnm)  under  the  assumption  that  Mi  =  M2  =  M. 
Although  the  examples  we  provide  from  Chapter  III  are  all  generalized  optimal  control 
problems,  it  is  worth  noting  that  the  results  of  this  chapter  are  applicable  to  other 
types  of  problems  as  well.  The  results  may  be  applicable  to  approximation  problems 
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for  (P)  that  involve  a  single  discretization  parameter  for  the  space  of  decision  variables 
and  two  discretization  parameters  for  the  objective  function. 

To  solve  the  infinite- dimensional  problem  (P)  some  form  of  discretization  is 
necessary.  If.  for  example,  the  infinite-dimensional  problem  takes  the  form  of  ( GTP ) 
and  N  and  M  represent  time  and  space  discretization  levels,  respectively,  then  the 
discretization  in  time  of  the  searcher’s  dynamics  using  Euler’s  method  and  approxima¬ 
tion  of  spatial  integration  via  Simpson’s  rule  described  in  Section  III. B. 3b  combined 
with  a  standard  nonlinear  programming  algorithm  could  be  used  to  find  a  solution 
to  (P).  We  refer  to  this  type  of  solution  methodology  as  a  discretization  algorithm. 

In  order  to  solve  (P)  using  a  discretization  algorithm,  there  is  a  fundamental 
trade-off  between  the  level  of  discretization  and  the  computational  work  required  to 
approximately  solve  the  resulting  finite-dimensional  problem  ( Pjvm )•  A  fine  level  of 
N  and  M  discretization  ensures  that  ( Pnm )  approximates  (P),  in  some  sense,  with 
a  high  degree  of  accuracy.  The  issue  with  this  approach  is  that  the  function  and 
gradient  evaluations  in  (Pnm)  tend  to  be  expensive,  and  therefore  the  computational 
work  required  is  high.  A  lower  level  of  N  and  M  discretization  results  in  a  faster 
solution  time,  due  to  the  relatively  less  expensive  function  and  gradient  evaluations, 
but  at  the  expense  of  a  less  accurate  approximation  of  (P).  It  is  usually  difficult  in 
practice  to  effectively  manage  the  trade-off  between  these  diametrically  opposed  goals 
of  solution  accuracy  and  computational  cost. 

In  this  chapter,  we  investigate  the  rate  of  convergence  of  a  class  of  discretiza¬ 
tion  algorithms  as  a  computing  budget  tends  to  infinity.  We  show  that  the  policy  for 
selecting  N  and  M  discretization  levels  relative  to  the  size  of  the  available  computa¬ 
tional  budget  influences  the  rate  of  convergence  of  discretization  algorithms.  We  iden¬ 
tify  optimal  discretization  policies,  in  a  specifically  defined  sense,  for  discretization 
algorithms  used  to  solve  the  resulting  finite-dimensional  problems  based  on  finitely, 
superlinearly,  linearly,  and  sublinearly  convergent  optimization  algorithms. 

Other  researchers  have  examined  the  rate  of  convergence  for  optimization  prob- 


lems  where  discretization  is  necessary  to  obtain  solutions.  Dunn  and  Sachs  (1983) 
considers  the  effect  of  approximate  function  and  gradient  evaluations  on  the  con¬ 
vergence  rates  of  the  conditional  gradient  and  projected  gradient  optimization  algo¬ 
rithms.  Kelley  and  Sachs  (1986)  examines  the  effect  of  approximating  the  Hessian  on 
the  convergence  rate  of  the  BFGS-method  when  it  is  used  to  solve  discretized  opti¬ 
mal  control  problems.  Dupuis  and  James  (1998)  presents  a  method  for  obtaining  rate 
of  convergence  estimates  for  finite  difference  approximation  schemes  for  stochastic 
and  deterministic  optimal  control  problems.  The  method  given  in  Dupuis  and  James 
(1998)  requires  some  assumption  about  the  smoothness  of  the  objective  function  be¬ 
ing  approximated,  which  they  exploit  to  obtain  sharp  rates  and  in  some  cases  an 
expansion  of  the  discretization  error  in  terms  of  the  discretization  parameter.  Kang 
(2008)  gives  rate  of  convergence  results  for  the  approximate  optimal  cost  computed 
when  using  pseudospectral  methods  to  solve  optimal  control  problems.  These  studies 
all  share  two  important  similarities.  The  first  being  that  the  approximation  schemes 
under  consideration  are  all  based  on  a  single  discretization  parameter,  and  the  sec¬ 
ond  being  that  none  of  the  studies  consider  the  error  introduced  by  the  optimization 
algorithm  in  their  determination  of  the  rate  of  convergence. 

More  recent  studies  in  the  Monte  Carlo  simulation  and  simulation  optimization 
literature  (see  Chen  &  Shi,  2008  for  a  summary)  deal  with  how  to  optimally  allocate 
a  computational  budget  across  different  tasks  within  the  simulation  and  to  determine 
the  resulting  rate  of  convergence  of  an  estimator  as  the  computational  budget  tends  to 
infinity.  In  the  stochastic  optimization  literature,  Pasupathy  (2010)  and  Royset  and 
Szechtman  (2011)  consider  optimization  algorithms  with  sublinear,  linear,  and  super- 
linear  rates  of  convergence  for  use  with  the  sample  average  approximation  approach, 
determine  optimal  policies  for  allocating  a  compuational  budget  between  sampling 
and  optimization,  and  quantify  the  associated  rate  of  convergence  of  the  sample  av¬ 
erage  approximation  approach  as  the  computational  budget  tends  to  infinity.  In  the 
semi-infinite  minimax  literature,  Royset  and  Pee  (2011)  examines  the  convergence 


rate  as  the  computing  budget  tends  to  infinity,  and  provide  allocation  polices  that 
maximize  the  convergence  rate  of  sublinear,  linear,  superlinear,  and  finite  algorithms 
used  to  solve  semi-infinite  minimax  problems.  Royset  and  Pee  (2011)  also  examines 
the  solution  of  the  semi-infinite  minimax  problem  by  exponential  smoothing  algo¬ 
rithms,  provides  an  optimal  discretization  and  smoothing  policy,  and  determines  the 
corresponding  rate  of  convergence  as  the  computing  budget  tends  to  infinity.  We  note 
that  the  section  on  smoothing  algorithms  in  Royset  and  Pee  (2011)  uses  two  distinct 
discretization  parameters.  Our  approach  in  this  chapter  follows  the  methodology  of 
Royset  and  Szechtman  (2011)  and  Royset  and  Pee  (2011).  Our  analysis  is  based  on 
different  assumptions,  however,  and  as  a  result  we  reach  different  conclusions. 

The  organization  of  this  chapter  is  as  follows.  We  begin  by  defining  terminol¬ 
ogy  and  presenting  our  assumptions.  Next  we  consider  finite,  superlinear,  linear,  and 
sublinear  optimization  algorithms  for  solving  the  finite-dimensional  problem  ( Pnm )• 
For  each  class  of  optimization  algorithm  we  give  an  optimal  discretization  policy  with 
corresponding  rate  of  convergence,  as  the  computational  budget  tends  to  infinity. 
Finally,  we  state  our  conclusions. 


B.  TERMINOLOGY  AND  ASSUMPTIONS 


We  begin  with  some  assumptions  about  (P)  and  (Pnm)- 
Assumption  IV.  1.  We  assume  that  the  following  hold: 

(i)  The  set  of  optimal  solutions  X*  of  (P)  is  nonempty. 

(ii)  There  exist  constants  N,  M  G  N,  and  K  G  [0,  oo)  such  that 

(a)  the  set  of  optimal  solutions  X^M  of  (Pnm)  is  nonempty  for  all  N  >  N, 
M  >  M,  N,  M  G  N,  and 

(b)  there  exist  p,  q  G  (0,  oo)  such  that 


\f(x)  ~  fNM(x)\  <  ^  + 


(IV.3) 


for  all  x  G  XN,  N  >  N,  M  >  M,  and  N,  M  G  N. 
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If,  for  example,  ( Pnm )  is  a  generalized  optimal  control  problem  of  the  form  defined  in 
Chapter  III,  then  part  (b)  of  item  (ii)  holds  when  (Pnm)  is  solved  using  an  algorithm 
that  utilizes  Euler’s  method  to  approximate  the  solutions  of  the  differential  equations 
and  approximates  spatial  integration  using  Simpson’s  rule.  In  this  case  p  =  1  and 
q  =  4;  see  Proposition  III. 26  and  Proposition  III. 50.  We  refer  to 

f(x)  -  f nm(x )  (IV.4) 

as  the  discretization  error.  Unless  Xn  and  / nm(x ),  x  G  Xn,  have  special  structures,  it 
is  impossible  to  obtain  a  globally  optimal  solution  of  (Pnm)  in  finite  computing  time. 
Therefore,  once  a  finite  number  of  iterations  of  an  optimization  algorithm  are  applied 
to  (Pnm),  there  is  usually  optimization  error.  Given  an  optimization  algorithm  A 
for  (Pnm),  let  xf,M  6  Xjv  be  the  iterate  obtained  by  A  following  n  iterations  on 
(Pnm)-  We  use  the  notation  to  denote  the  optimal  value  of  (Pnm)  and  define 
the  optimization  error  as 

f N  m(An  M)  —  f NM-  (IV. 5) 

The  rate  of  decay  of  the  optimization  error  as  n  gets  larger  depends  on  the  rate  of 
convergence  of  A  when  used  to  solve  (Pnm)-  Throughout  this  dissertation  we  only 
consider  deterministic  algorithms  that  generate  iterates  in  A"  exclusively,  which  we 
formalize  in  the  next  assumption. 

Assumption  IV. 2.  We  assume  that  if  {x nm}ti=o>  N,  M  E  N,  are  generated  by  a 
given  optimization  algorithm  when  applied  to  (Pnm),  then  xr}^M  E  X  for  all  N,  M  gN 
and  n  =  0, 1,  2, ....  □ 

Based  on  the  definition  of  X  in  Section  III. B. 3a,  many  optimization  algorithms 
satisfy  Assumption  IV.2.  We  use  the  notation  /*  to  denote  the  optimal  value  of  (P) 
and  define  the  total  error  as 

I/Km)- A  (IV. 6) 
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which  is  a  measure  of  the  quality  of  the  solution  obtained  after  applying  n  iterations 
of  A  to  ( Pnm )■  Based  on  Assumptions  IV. 1  and  IV. 2, 


I  f{x 


n  \ 
NM) 


~  f*  I  —  I/(x7Vm)  —  /wm(^jvm)  +  /wm(^jvm)  —  /iVM  ~  f*  +  /nm\ 

*  Tv +  S? +  (IV'7) 


where  A^fM(A)  is  an  upper  bound  on  the  optimization  error  after  n  iterations  of 
optimization  algorithm  A  are  applied  to  (Pnm)-  In  this  chapter  we  consider  several 
different  expressions  for  A^fM(A)  under  various  assumptions  about  the  optimization 
algorithm,  and  hence  also  about  (Pnm)-  Because  it  appears  difficult  to  quantify  the 
rate  of  convergence  of  the  total  error,  as  in  Royset  and  Pee  (2011)  we  concentrate  on 
the  rate  of  convergence  of  its  upper  bound  in  (IV.7).  From  (IV. 7),  it  is  clear  that  the 
rate  of  convergence  of  that  bound  gives  a  guaranteed  minimum  rate  of  convergence 
of  the  total  error. 

Based  on  (IV.7),  it  is  evident  that  different  choices  of  N,  M,  and  n  may 
result  in  different  bounds  on  the  total  error.  Let  b  G  N  be  the  computational  budget 
available  to  apply  n  iterations  of  A  to  (Pnm)-  To  make  it  clear  that  the  choice  of 
N,  M ,  and  n  would  typically  depend  on  6,  we  write  IV&,  Mb ,  and  rib-  We  refer  to 
{(nb,  Nb,  Mb)}^=l,  with  rib,Nb,Mb  G  Id  for  all  b  G  N,  as  a  discretization  policy.  A 
discretization  policy  specifies  the  level  of  discretization  of  (P)  as  well  as  the  number 
of  iterations  of  the  optimization  algorithm  to  complete  for  any  computational  budget. 
If  rib,  Nb,  Mb  — *  oo,  as  b  — »  oo,  then  based  on  Assumption  IV. 1,  the  bound  on  the 
discretization  error  vanishes.  If  we  assume  that  a  convergent  optimization  algorithm 
is  applied  to  solve  (Pnm),  then  the  optimization  error  and,  hence,  the  corresponding 
bound  could  vanish  as  well.  For  a  given  optimization  algorithm  A,  and  n ,  JV,MgN, 
we  define  the  total  error  bound,  denoted  by  e(A,  n,  N,  M),  as  the  right-hand  side  of 
(IV.7)  and  have 

e(A,  n,  N,  l»)44  +  4).  (IV.8) 


92 


In  this  chapter,  we  investigate  the  rate  at  which  the  total  error  bound  e(A,  nb,  Nb,  Mb) 
vanishes  as  b  tends  to  infinity  for  different  discretization  policies  {( nb ,  Nb,  and 

optimization  algorithms  A.  We  give  optimal  discretization  policies,  which  as  discussed 
in  detail  below,  attain  the  highest  possible  rate  of  convergence  of  the  total  error 
bound  as  the  computational  budget  tends  to  infinity  for  a  given  class  of  optimization 
algorithms. 

The  analysis  that  follows  relies  on  the  following  assumption  about  the  com¬ 
putational  work  needed  by  an  optimization  algorithm  to  complete  n  iterations  on 
( Pnm )• 

Assumption  IV. 3.  There  exist  a  a  =  a  (A)  G  (0,oo),  a  p  =  p(A)  G  (0,  oo),  and  a 
v  =  v(A)  G  (0,  oo)  such  that  the  computational  work  required  by  a  given  optimization 
algorithm  A  to  complete  n  G  N  iterations  on  (Pnm),  N,  M  E  N,  is  no  larger  than 
anN^M”.  □ 

Assumption  IV. 3  holds  with  p  —  1  and  v  —  2  if  the  optimization  algorithm  A  is 
a  steepest  descent  method.  This  is  because  each  iteration  of  these  algorithms  requires 
the  calculation  of  /nm(x)  at  the  current  iterate  x  G  XN,  as  well  as  the  evaluation 
of  the  gradient  V /nm(x),  which  dominates  the  computational  budget.  In  order  to 
evaluate  the  gradient,  using  problem  (GTP^M)  from  Chapter  III  as  an  example, 
we  see  from  (III. 60)  that  0(N)  operations  are  required  to  find  the  physical  states, 
while  (III. 61)  and  (III. 73),  respectively,  indicate  that  0(NM 2)  operations  are  required 
to  find  the  information  state  and  adjoint.  This  leads  to  an  overall  computational 
complexity  of  0(NM 2)  to  evaluate  V/jvm(^)- 

The  SQP  algorithm  in  the  TOMLAB  SNOPT  solver  (see  Gill  et  ah,  2007)  also 
seems  to  empirically  follow  Assumption  IV. 3  with  p  =  1  and  v  —  2  when  used  to  solve 
(i GTPC )  with  K  =  1  and  L  —  10  using  Algorithm  V.2,  which  will  be  discussed  further 
in  Chapter  V,  with  fjo  =  (7r/4,  0  G  M^0),  target  and  HVU  parameters  values  given  in 
Table  6  (except  vmax  =  60),  and  searcher  and  detection  rate  parameters  given  in  the 
k  —  1  columns  of  Tables  7  and  8,  respectively.  Table  1  shows  the  computational  time 
in  seconds  required  to  complete  five  iterations  of  the  SQP  algorithm  for  various  values 
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of  N  and  M .  Table  1  also  shows  the  fitted  values  for  a  zero  intercept  linear  regression 
done  in  R,  version  2.13.0  (see  Team,  2010),  for  the  linear  model  Y  =  aNM'2.  Figure 
2  illustrates  the  close  agreement  between  the  actual  and  fitted  values  given  in  Table 
1. 


Table  1.  Actual  and  fitted  computational  time  in  seconds  for  five  iterations  of  the 
SQP  algorithm  in  the  TOMLAB  SNOPT  solver. 


Instance 

N 

M 

Actual 

Fitted 

1 

16 

7 

1.854 

1.427 

2 

16 

9 

2.164 

2.359 

3 

32 

7 

2.620 

2.855 

4 

32 

9 

4.176 

4.719 

5 

64 

7 

4.532 

5.709 

6 

64 

9 

8.655 

9.438 

7 

128 

7 

10.152 

11.419 

8 

64 

11 

13.337 

14.099 

9 

128 

9 

15.279 

18.876 

10 

128 

11 

24.154 

28.197 

11 

128 

15 

42.168 

52.433 

12 

128 

17 

60.439 

67.348 

13 

256 

11 

69.144 

56.395 

14 

256 

15 

102.924 

104.866 

15 

256 

17 

120.854 

134.695 

16 

512 

15 

196.666 

209.733 

17 

512 

17 

289.129 

269.390 

In  some  implementations,  particularly  those  involving  discretization  of  opti¬ 
mal  control  problems  using  collocation  methods,  increasing  the  time  discretization 
parameter,  N,  gives  rise  to  a  large  number  of  equality  constraints.  This  large  number 
of  equality  constraints  provides  special  structure  that  allows  for  sparse  implementa¬ 
tions  where  the  computational  work  may  grow  slowly  as  N  increases.  Assumption 
IV. 3  may  account  for  the  slow  growth  of  the  computational  work  in  N,  by  selecting 
a  value  for  /i  e  (0, 1). 

Based  on  Assumption  IV. 3,  as  in  Royset  and  Pee  (2011),  we  refer  to  a  dis¬ 
cretization  policy  {(nb,  Nb,  Mb)}(^=1  as  asymptotically  admissible  if  anbNjfM£ /b  — *  1, 
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Problem  Instance 


-Actual 


-•—Fitted 


Figure  2.  Regression  fit  for  Y  =  aNM 2  model. 

as  b  — »  oo.  It  is  obvious  that  an  asymptotically  admissible  discretization  policy  satis¬ 
fies  the  computational  budget  in  the  limit  as  b  tends  to  infinity.  In  the  next  section,  we 
determine  optimal  asymptotically  admissible  discretization  policies  and  correspond¬ 
ing  rates  of  convergence  of  the  total  error  bound  under  different  assumptions  about 
the  optimization  algorithm  and,  therefore,  the  optimization  error  bound  A 

C.  RATE  ANALYSIS  FOR  CLASSES  OF  ALGORITHMS 

From  (IV. 7)  we  see  that  the  total  error  bound  consists  of  discretization  and 
optimization  error  bounds.  The  discretization  error  bound  depends  on  the  discretiza¬ 
tion  levels  N  and  M,  but  not  on  the  optimization  algorithm  used.  The  optimization 
error  bound  depends  on  the  rate  of  convergence  of  the  optimization  algorithm  used 
to  solve  ( Pnm )•  In  this  section,  we  consider  four  cases:  First,  we  assume  that  the 
optimization  algorithm  solves  (Pnm)  in  a  finite  number  of  iterations.  Second,  we 
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investigate  optimization  algorithms  with  a  superlinear  rate  of  convergence  towards 
an  optimal  solution  of  ( Pnm )•  Third,  we  consider  linearly  convergent  optimization 
algorithms.  Fourth,  we  deal  with  sublinearly  convergent  algorithms. 

1.  Finite  Optimization  Algorithm 

Suppose  that  the  optimization  algorithm  used  to  solve  (Pnm)  is  guaranteed  to 
find  an  optimal  solution  in  a  finite  number  of  iterations,  independently  of  N  and  M. 
We  define  finitely  convergent  algorithms  based  on  the  following  definition  adapted 
from  Royset  and  Pee  (2011).  We  note  that  our  definitions  for  superlinearly,  linearly, 
and  sublinearly  convergent  algorithms  are  also  adapted  from  Royset  and  Pee  (2011). 

Definition  IV.  1.  An  optimization  algorithm  A  converges  finitely  on  {( Pnm )}n^  m=m 
when  X*NM  is  nonempty  for  N  >  N,  M  >  M,  and  there  exists  a  constant  n  G  N  such 
that  for  all  N  >  N,  M  >  M,  N,  M  e  N,  a  sequence  {xnm}™=o  generated  by  A  when 
applied  to  (Pnm)  satisfies  x^M  G  X*NM  for  all  n  >  n.  □ 

No  optimization  algorithm  converges  finitely  on  {(Pnm)}™’^  m=m  without 
strong  structural  assumptions  on  Xn  and  f nm(- ),  such  as  linearity.  In  this  disserta¬ 
tion,  we  are  not  concerned  with  instances  of  (Pnm)  that  are  linear  programs,  which 
may  allow  for  finite  convergence.  We  include  this  case  here  to  serve  as  an  “ideal”  case. 

It  will  be  shown  below  that  this  case  gives  an  upper  bound  on  the  rate  of  conver¬ 
gence  of  the  total  error  bound  using  any  optimization  algorithm.  Based  on  Definition 
IV. 1,  a  finitely  convergent  optimization  algorithm  *4.fimte  on  {(Pnm)}^’^  m=m  ^ias  no 
optimization  error  after  performing  a  large  enough  number  of  iterations.  We  define 
AWAfimte)  =  0  and  e(„4.fimte,  n,  N,  M)  =  X-  _|_  JL  for  n  >  n,  N  >  N,  and  M  >  M, 
where  K  is  as  in  Assumption  IV.  1  and  n,  N,  and  M  are  as  in  Definition  IV.  1.  The 
next  theorem  gives  the  rate  of  convergence  of  the  total  error  bound  for  this  case. 

Theorem  IV. 4.  Suppose  that  Assumption  IV.  1  holds  and  that  A&mte  is  a  finitely  con¬ 
vergent  algorithm  on  {(Pnm)}°^’=n  m=m>  w ^  ^  an ^  ^  as  Assumption  IV.  1  and 
number  of  required  iterations  n  as  in  Definition  IV.  1.  Suppose  also  that  satis¬ 

fies  Assumptions  IV. 2  and  IV. 3.  If  {(n^,  Nb,  Mb)}ffi=1  is  an  asymptotically  admissible 
discretization  policy  with  nb  =  n  for  all  b  G  N,  then 
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1 


lim  inf  log  e(^4finite,  nb,  Nb,  Mb)  > 


(IV. 9) 


b^o O  log  b  (/i/p  H-  v/ q)  ’ 

where  p  and  q  are  as  in  Assumption  IV.  1,  and  p  and  v  are  as  in  Assumption  IV. 3. 
Furthermore,  if  Nb/hl^'1+pu/q)  — >•  Gq  e  (0,  oo)  as  b  — >■  oo;  i/ien 

log  e(.4finite,  nb,  Nb,  Mb)  1 


lim  inf 

6— >-oo 


(IV.10) 


log  6  (p/p  +  v/q)' 

Proof.  For  sufficiently  large  b,  A^rM(v4fimte)  =  0  and  e(A&mte,n,  N,  M)  =  -N  +  -N-, 
where  K  is  as  in  Assumption  IV.  1.  Consequently,  for  sufficiently  large  b , 


log  e(^lfimte,  nb,  Nb,  Mb)  =  log 

,  (  f  K  K 

>  log  max  {  -y,  tto 


(IV.ll) 


V 


nv  mi 


=  max  l  log  log 


K 


Mqb 


(IV.12) 


Hence, 

log  e(^4fimte,  nb,  Nb,  Mb) 
log  b 


>  max 


-p  log  Nb  +  log  K  —q  log  Mb  +  log  K 1 


log  b 


log  b 


I 


.  (IV.  13) 


We  now  consider  two  cases,  one  for  both  of  the  terms  inside  the  max  function  of 
(IV.  13)  when  it  is  greater  than  or  equal  to  the  other  term  inside  the  max  function. 
Initially,  we  want  the  first  term  to  be  greater  than  or  equal  to  the  second  term. 


This  will  be  true  if 

P log N b  >  -q\ogMb. 

(IV.  14) 

We  note  that  (IV.  14)  implies 

Ml  >  Nlu/q. 

(IV.15) 
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If  we  then  use  the  bound  on  Mb  obtained  from  (IV.  15),  we  have  that 


log  e(^Ifinite,  nb,  Nb,  Mb) 
log  b 


> 


p  log  Nb  +  log  K 
log  b 

-p  log  Nb  +  log  K 


log 


loS  (vhvfAf  J  +  lQg  {vnN£MZ 

-p  log  Nb  +  log  K 

los  +  log(ari)  +  log  [N* 


pv+qpL 
Q 


-p 


log  K 
log  At, 


(IV.16) 


log  Nb 


log  (an)  ,  pv+qp 

log  At,  ^  q 


Next,  we  consider  the  case  where  the  second  term  inside  the  max  function  of 
(IV.  13)  is  greater  than  or  equal  to  the  first  term  inside  the  max  function.  This  will 
be  true  if 


-q  log  Mh  >  -p\ogNb. 


(IV.  17) 


We  note  that  (IV. 17)  implies 


K  >  M^/p. 


(IV.18) 


If  we  then  use  the  bound  on  Nb  obtained  from  (IV.18),  we  have 
log  e(*4.fimte,  nb,  Nb,  Mb)  -q  log  Mb  +  log  K 


log  b 


> 


log  b 

-q  log  Mb  +  log  K 


> 


loS  )  +  loS  {™N£M'h 

-q  log  Mb  +  log  K 


log 


anNgM£ 


+  log  (cm)  +  log  (  Mt 


pv+qp 


-Q  +  l°sK 

1  ^  log  Mb 


log 


(IV.19) 


lNb  Mb  J  I  log(g-n)  ,  pu+qp 
log  Mb  log  Mb  p 


log  e(^4fimte,  nbl  Nb,  Mb) 
log  b 


We  now  consider 


as  b  — *  oo.  Since  anNjfMb /b  — *  1  as  b  — >  oo,  we  consider  two  cases:  one  where  at 
least  one  of  the  parameters  Nb  or  Mb  does  not  increase  without  bound  as  b  — *  oo, 
and  another  where  Nb  — >  oo  and  Mb  — *  oo  as  6  — *  oo.  For  the  first  case,  at  least 
one  of  the  parameters  Nb  or  Mb  remains  finite  as  b  — y  oo.  Then  based  on  (IV. 11), 
e(*4.fimte,  nb,  Nb,  Mb)  G  ^0,  lim  sup^^  ,  and  we  have  that 


log  e(*4.finite,  nb,  Nb,  Mb)  1 

hm  inf  - : - - -  >  0  > 

ft-s-oo  logo  (P/P+V/q) 


(IV.20) 


For  the  second  case,  where  both  of  the  parameters  Nb  and  Mb  increase  without  bound 
as  b  — »  oo,  we  have  that 


lim  inf  log  e(A&nite,nb,Nb,Mb) 

b- 5>oo  log  6 


>  min  <  hm  inf 

b— >oo 


-v+  lo&K 
y  ^  log  Nf, 


anN^M” 
b  b 


log  Nb 


_i_  log(tm)  ,  pu+qfj. 

log  -N,  "r  9 


lim  inf 

6— >-oo 


=  mm 


log  X 
log  M;, 


anN^Mt 
_ b  b 


log  Mb 

1 


log(tm)  pu+qp 
'  logM,,  "l"  p 


(p/p  +  v/qY  (fi/p  +  v/q) 


C p/p  +  v/qY 


which  completes  the  proof  for  the  first  part  of  the  theorem. 

Next,  let  {(rib,  Nb,  Mb)}^=1  be  an  asymptotically  admissible  discretization  pol¬ 
icy  with  rib  =  n  satisfying  Vj/Zi'/V+W?)  — >  ai  G  (0,  oo)  as  6  — »  oo.  For  notational 
simplihcation  we  dehne 


i  — 


1 

Kb  v/v+v /q 

ivp 


(IV.21) 
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Then,  for  sufficiently  large  b ,  e(^4fiiute,  n,  N,  M)  =  and  algebraic  manipulation 

gives 


e(Afinite,  nb,  Nb,  Mb) 


b  \<t/v 


TV"  /  b  \ 

=  Bi  5m/p+^/9  -I - V  anN ^  J 

'  b  \<l!v  Ml 


-l 

=  b^ivJrViq 


V  crnN v  J 

(  b  Kcrqlv'nqlv  1 

B1  + 


Mi 


bi/h 


Algebraic  manipulation  can  be  used  to  show  that 


(IV.22) 


,  1  n 

Jjpv+pv2/q  Q/l/fo  p/p+v/q  =  =  1. 


(IV.23) 


Then,  since  aribNjf  M£ /b  — y  1,  the  sum  of  terms  in  brackets  in  (IV.22),  with  n,  N, 
and  M  replaced  by  n,  Nb,  and  Mb,  respectively,  tends  to  a  constant  as  b  — »  oo.  The 
conclusion  for  the  second  part  of  the  theorem  then  follows  from  (IV.22)  after  taking 
logarithms,  dividing  by  log  b,  and  taking  limits.  □ 

Theorem  IV. 4  shows  that  e(Afimte,  rib,  Nb,  Mb)  converges  at  a  rate  fr-VO/p+V?) 
under  the  stated  discretization  policy.  From  (IV.8)  we  see  that  the  total  error  bound 
includes  the  discretization  error  bound.  This  means  the  total  error  bound  cannot 
converge  faster  than  the  rate  ^VO/p+Vk)  no  matter  which  optimization  algorithm  is 
used. 

It  would  be  difficult  to  implement  the  asymptotically  admissible  discretization 
policy  stated  in  Theorem  IV. 4  because  n  may  be  unknown.  Despite  this  difficulty, 
the  rate  of  convergence  obtained  in  Theorem  IV. 4  is  useful  as  an  upper  bound  on 
the  rate  that  can  be  achieved  by  any  optimization  algorithm,  and  will  be  used  as  a 
benchmark  for  comparison  in  the  case  of  superlinear,  linear,  and  sublinear  algorithms 
below. 

2.  Superlinear  Optimization  Algorithm 

We  will  now  consider  superlinearly  convergent  optimization  algorithms.  We 
define  superlinearly  convergent  algorithms  based  on  the  following  definition. 
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Definition  IV.  2.  An  optimization  algorithm  A  converges  superlinearly  with  order 
7  G  (l,oo)  on  {(Pnm)}^^  m=m  when  i s  nonempty  for  N  >  N,  M  >  M,  and 

there  exist  constants  n  G  N,  c  G  [0,  oo),  and  p  G  [0, 1)  such  that  c1 Aw 1 } {JnmF'nm)  ~ 
f*NM )  <  P  and 


Jnm  (^jym)  Inm 

JnM  ( XNM )  —  /atm]7 


<  c, 


(IV.24) 


for  all  n  >  n,  n  G  N,  N  >  N,  and  M  >  M ,  N,  M  G  N. 


□ 


Definition  IV. 2  requires  that  the  optimization  algorithm  achieve  a  superlinear 
rate  of  convergence  for  sufficiently  large  n.  This  is  usually  the  case  with  7  =  2 
for  Newtonian  methods  applied  to  strongly  convex  instances  of  ( Pnm )  with  twice 
Lipschitz  continuously  differentiable  objective  functions.  The  next  lemma  gives  a 
total  error  bound  for  a  superlinearly  convergent  algorithm. 


Lemma  IV. 5.  Suppose  that  Assumption  IV.  1  holds  and  that  yfsuPer  is  a  superlinearly 
convergent  algorithm  with  order  7  G  (1,  00)  on  {(Pnm)}^’^  m=m>  undh  N  and  M  as 
in  Assumption  IV.  1.  Let  {xr^M},^>=0  be  the  iterates  generated  by  yfsuPer  when  applied 
to  (Pnm),  NeN,  N>N,  Me  N,  M>  M .  Suppose  also  that  7fsuPer  satisfies 
Assumptions  IV. 2  and  IV. 3.  Then,  there  exist  constants  c  G  (0,1);  K  G  [0,  00),  and 
he  N  such  that 


f(x 


n 

NMJ 


-  F  <  Auk  + 


K 

Np  + 


K 

Wv 


(IV.25) 


for  all  n  >  n,  n  e  N,  N  >  N ,  N  e  N,  and  M  >  M,  M  G  N,  where  K ,  p,  and  q  are 
as  in  Assumption  IV.  1. 


Proof.  The  proof  follows  the  same  arguments  as  those  for  the  proof  of  Lemma  1  in 
Royset  &  Pee  (2011).  □ 

From  Lemma  IV. 5,  we  adopt  the  upper  bound  on  the  optimization  error 


A)W(*4supen  A 


)  —  C  K 


(IV.26) 


for  a  superlinearly  convergent  optimization  algorithm  7fsuPer  on  {(Pnm)}^F^  m=mi 
where  c  and  n  are  as  in  Lemma  IV.5.  Then  for  n,  N,  M  G  N,  we  define  the  total  error 


bound 


e(^4super,  n,  N,  M )  A  CP k  + 
y  ’  ’  ’  ’  Np  Mi 


(IV.27) 


101 


The  next  result  shows  that  if  we  select  a  particular  discretization  policy,  then  a  super- 
linearly  convergent  optimization  algorithm  results  in  the  same  rate  of  convergence  of 
the  total  error  bound  as  a  finitely  convergent  algorithm.  Therefore,  the  policy  speci¬ 
fied  in  the  following  theorem  is  optimal  in  the  sense  that  no  other  policy  guarantees 
a  better  rate  of  convergence. 


Theorem  IV. 6.  Suppose  that  Assumption  IV.  1  holds  and  that  *4.super  is  a  superlin- 
early  convergent  algorithm  of  order  7  G  (1,  00)  on  {(Pjvm)}^^  m=m>  with  N  and 
M  as  in  Assumption  IV.  1.  Suppose  also  that  *4.super  satisfies  Assumptions  IV. 2  and 
IV. 3.  If  {(nb,  Nb,  Mb)}(£L1  is  an  asymptotically  admissible  discretization  policy  with 


or 


and 


then 


^ftlogy  pg 

— ^  cl\  —  1  and  - ; - <  1, 


log  (  Xi, 

nb  log  7 


log  3 


log  b 


-  logo 


Nh 


6  log  7 


r  log  (  ~ 


log  b 


log  c  > 


pq  +  vp 


— y  Oi  G  (1, 00) , 


— y  q-2  c  (0, 00), 


pq+vp 


lim 

b— >00 


log  e(^4super,  nb,Nb,  Mb) 


(IV.28) 


(IV.29) 


(IV.30) 


,  ,  -  -1—, - mi  (IV.31) 

logo  {P/P  +  v/q) 

where  p  and  q  are  as  in  Assumption  IV.  1,  and  p  and  v  are  as  in  Assumption  IV. 3. 


Proof.  For  notational  simplification  we  define 


and 


n  log  'y 


Bi=ex  p|logK  +  logc[TVV”-(^ 

log  c 


K 


Bo  = 


b  log  7 


rl°g(d 


log  b 


log  c  , 


pq 

pq+vp 


Np 


B3 


b  log  7 


-pq 

pq+vp 


(IV.32) 


(IV.33) 


(IV.34) 
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Algebraic  manipulation  of  (IV. 27)  gives 


e(Asuper,nb,Nb,Mb) 

/  n  log  7 

=  exp  I  log  k  +  ")/los(  -iogc) 


log  (  logt)  ' 

10&7V-logc, 


logc 


—  B\  +  B2B3  + 


K 

(_b_\q/v 

V  anN v  ) 


b  \<i/v 

anN P  ) 

Mi 


+  B2B3  + 


K 

(  b  \q/v 
\  anN v  ) 


(  b  y/v 

V  anN t*  ) 

Mi 


=  b3 


B 1 

B3 


+  b2  + 


crnN A4  )9/  Kaq/1'nq/vNilq/v 


Mq 


bq!v 


B, 


(IV.35) 


We  now  focus  on  the  following  portion  of  the  first  term  inside  the  square 
brackets  in  (IV.35), 

/ 

log  b 


Blb^+vp  =  /texp  logc 


logc 


log 


n  log  7 

log  b 

—  log  c 


pq 

Qpq+vp 


(IV.36) 


To  simplify  the  analysis,  we  take  the  logarithm  and  obtain 

pq 


log  hi  +  log  c 

=  log  K  +  log  b 
Since  anbN^M^ / b  — »  1, 


n  log  7 

log  b  \ 


logc 


—  log  c  J 


+ 


log  b 


pq  +  up 

n  log  7 _ 


l0SC,..„  (logt)1-^)"  + 


pq 


(—  log  c)  log(-°°gc) 


pq  +  up 


(IV.37) 


nb  log  7 


Nb 


b  log  7 

CTl°s(^f Sh) 


q 

pq+up 


— >  Q 1 , 


— )■  a2, 


and,  due  to  the  facts  that  logc  <  0  and  a3  =  1  and  ^  <  1  or  a  1  G  (l,oo),  the 

expression  in  (IV.37)  goes  to  —00,  as  b  — »  00,  we  see  that  the  sum  of  terms  in  brackets 
in  (IV.35),  with  n,  N,  and  M  replaced  by  nb,  Nb,  and  Mb,  respectively,  tends  to  a 
constant  as  b  — »  00.  The  conclusion  then  follows  from  (IV.35)  after  taking  logarithms, 
dividing  by  log  b,  and  taking  limits.  □ 
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While  no  discretization  policy  can  guarantee  a  rate  of  convergence  better  than 
b~1/^/p+u/q) ,  it  is  possible  to  do  worse.  If.  for  example,  {(rib,  Nb,  Mb)}bLi  is  an  asymp¬ 
totically  admissible  discretization  policy  with 


nb  ,  ,  P<1  ^ 

— »  oi  =  1  and  - ; -  <  1, 


b1/ 2 


/ug  +  zzp 


(IV.38) 


or 


and 


rib 

b1/ 2 

A, 


bV2 


— ^  Oi  G  (1,  oo), 


— ^  Cl 2  G  (0,  oo), 


pq+up 


(IV.39) 


(IV.40) 


then  it  can  be  shown  using  arguments  similar  to  those  used  in  the  proof  of  Theorem 
IV. 6  that 

\oge(Asuper,nb,Nb,Mb)  1 


lim 

6— >-oo 


log  b 


2/z  ■  2v 
P  Q 


(IV.41) 


which  is  worse  than  the  optimal  rate  by  a  factor  of  two. 


3.  Linear  Optimization  Algorithm 

We  now  consider  linearly  convergent  optimization  algorithms.  We  define  lin¬ 
early  convergent  algorithms  based  on  the  following  definition. 


Definition  IV. 3.  An  optimization  algorithm  A  converges  linearly  on  {( Pnm m=m 
when  X*N  is  nonempty  for  N  >  N,  M  >  M  and  there  exist  constants  n  G  N  and 
c  G  (0, 1)  such  that 


Jnm  (xnm)  /. 


NM 


<  c, 


f NM  ( XNM )  /iVM 

for  all  n  >  n,  n  G  N,  N  >  N,  N  G  N,  and  M  >  M ,  M  G  N. 


(IV.42) 

□ 


Definition  IV. 3  slightly  extends  a  standard  definition  of  linear  convergence  to 
require  that  the  rate  of  convergence  coefficient  holds  for  all  N  and  M  sufficiently  large. 
This  is  satisfied  by  many  gradient  methods,  such  as  the  steepest  descent  method  and 
projected  gradient  method,  when  applied  to  (Pnm)  under  the  assumption  that  the 
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objective  function  is  strongly  convex  and  twice  continuously  differentiable  and  that 


X  is  convex.  The  next  lemma  gives  a  total  error  bound  for  a  linearly  convergent 
algorithm. 

Lemma  IV. 7.  Suppose  that  Assumption  IV.  1  holds  and  that  yflmear  is  a  linearly 
convergent  algorithm  on  {(Pnm)}^'^  m=m>  m^1  N  and  M  as  in  Assumption  IV. 1. 
Let  {xnNM}™=Q  be  the  iterates  generated  by  *4linear  when  applied  to  ( Pnm ),  N  G  N; 
N  >  N ,  M  G  N,  M  >  M.  Suppose  also  that  there  exists  a  constant  C  G  M  such  that 
fNM{x%)  <  C  for  all  n  gN  ,  N  >  N ,  N  G  N,  and  M  >  M ,  M  G  N,  and  that  yflmear 
satisfies  Assumption  IV. 2.  Then,  there  exists  a  constant  k  G  [0,  oo)  such  that 


f{x 


n  \ 
NMJ 


-  f  *  <  CnK,  + 


K 

N p  + 


K 

Wv 


(IV.43) 


for  all  n  >  n,  N  >  N ,  and  M  >  M ,  where  c  and  n  are  as  in  Definition  IV.  3,  and 
K,  p,  and  q  are  as  in  Assumption  IV.  1. 


Proof.  Based  on  Assumption  IV.  1  and  the  fact  that  Alinear  is  linearly  convergent,  we 
obtain  that 


I f(xNM)  f  I  —  Inm(x %m)  +  jyp  +  ^.q  f NM 


<  cn~n 

<  c" 


n  *  K  K 

[ ./'jV A/f  (,Xj\j j  f N K'l\  jyp 


c 


K  K 


K  K 


C  ~  f*  +  ^  +  ^\  +  —  + - .  (IV.44) 

J  A  Tr,  T\  /T n  I  I  jy p  J^/Jq  V  ) 


Np  M\ 

Hence,  the  results  hold  with  k  =  c~n  (C  —  f*  +  +  ~j~).  □ 

From  Lemma  IV. 7,  we  adopt  the  upper  bound  on  the  optimization  error 


A”  M(-4linear)  = 


(IV.45) 


for  a  linearly  convergent  optimization  algorithm  *4.lmear  on  {(P/vm)}^^  m=m'  where 
c  and  k  are  as  in  Lemma  IV. 7.  Then  for  n,  N,  M  G  N,  we  define  the  total  error  bound 

n,N,M)&  ffV  +  A  +  V  (IV.46) 

The  next  result  shows  that  if  we  select  a  particular  discretization  policy,  then  a 
linearly  convergent  optimization  algorithm  can  also  attain  the  best  possible  rate  of 
convergence  of  the  total  error  bound  given  in  Theorems  IV.4  and  IV. 6. 


105 


Theorem  IV. 8.  Suppose  that  Ahnear  satisfies  the  assumptions  of  Lemma  IV.  7  and,  in 
addition,  Assumption  IV. 3  holds.  If  {(nb,  Nb,  Mb)}'jfl1  is  an  asymptotically  admissible 
discretization  policy  with 


-nb  log  c 
log  b 


— y  a i  £ 


PI 

pq  +  vp: 


(IV.47) 


and 


Nb 

q 

Hq+vp 


— 6  logo 
a  log  b 


— y  0.2  €  (0,  oo), 


(IV.48) 


where  c  is  as  in  Definition  IV. 3,  p  and  q  are  as  in  Assumption  IV.  1,  and  p  and  v  are 
as  in  Assumption  IV. 3,  then 


lim  log  e(v4Hnear,  rib,  Nb,  Mb)  _  1 

b—>oo  log  b  p/p  +  n/q 


(IV.49) 


Proof.  For  notational  simplification  we  define 


£?i  = 


K 


pq 

—b  log  c  p-q+vp 
<j  log  b 


Np 


and 


B2 


f  —  51ogc\ 
V  a  log  6  y 


-pq 

pq+vp 


Algebraic  manipulation  of  (IV.46)  gives 


(IV.50) 


(IV.51) 


Since 


e(AlmeaT,nb,Nb,Mb) 

(  —n log  c  log  6  ^  ,  D  d  ,  K  (vVm) 

exp  log  k  H - - — - - — =  log  c  +  BiB2  + 


b  \q/v 


log  b  —  log  c 


Tl  log  C 

exp  I  log  hi  +  - — —  log  6  )  +  BiB2  + 


(  b  y/v  Mi 

\  crnNP  ) 

K  (-^-)q/v 

\  anNv  ) 


—  re  log  c  +  B\B2 

b  los b 


log  b  )  1  z  (  b  yA 

\  crnNP  ) 

K  (-b-)q/v 

\  anN »  ) 


=  Bo 


K 


—  n  log  c 

B2b  l°sb 


B1 


(  b  y/v  M1* 

V  crnNP  ) 

(  b  y/ v  j£aq/vnq/v]ynq/v 


k  crnNP , 


B2M1 


bci/v 


-nb  log  c 
log  b 


— y  Gp  G 


pq 


pq  +  np 


,oo), 


(IV.52) 
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the  expression 


(IV.53) 


K 

—  n  log  c 

B2b  los 6 

goes  to  0,  as  b  — »  oo.  In  addition,  we  know  that  ant,NjfM£ /b  — *  1,  and 


Q 

Liq+vp 


— 61ogc 
(J  log  6 


— )■  02, 


as  b  — >■  oo.  Consequently,  the  sum  of  terms  in  brackets  in  (IV. 52),  with  n,  N,  and 
M  replaced  by  rib,  iVj,,  and  Mj,,  respectively,  tends  to  a  constant  as  b  — *  oo.  The 
conclusion  then  follows  from  (IV.52)  after  taking  logarithms,  dividing  by  log  b,  and 
taking  limits.  □ 


4.  Sublinear  Rate  of  Convergence 

Next  we  consider  sublinearly  convergent  optimization  algorithms.  We  define 

sublinearly  convergent  algorithms  based  on  the  following  definition. 

Definition  IV. 4.  An  optimization  algorithm  A  converges  sublinearly  with  degree 
7  G  (0,  oo)  on  {(TWm)}^^  m=m  when  is  nonempty  for  N  >  N,  M  >  M  and 
there  exists  a  constant  C  G  [0,  oo)  such  that 

f NM  ( XNM )  —  $NM  —  (IV.54) 

for  all  n  G  N,  N  >  N,  N  G  N,  and  M  >  M,  M  G  N.  □ 


The  subgradient  method  is  sublinearly  convergent  in  the  sense  of  Definition 
IV. 4  with  7  =  1/2  and  C  =  DxLf  when  ( Pnm )  is  convex,  where  Dx  is  the  diameter  of 
XN  and  Lf  is  a  Lipschitz  constant  of  / Am (•)  on  X ;  see  Nesterov  (2004),  pp.  142-143. 
Based  on  Definition  IV. 4,  we  define  the  optimization  error  bound  for  a  sublinearly 
convergent  optimization  algorithm  yfsublm  by 

A^„(.4-ub,i")  &  T  (IV.55) 

and  the  total  error  bound  for  n,  N,M  GN  by 

e(_4”“'\  a,  W,M)A  +  4  +  A  (IV. 56) 
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The  next  theorem  gives  an  optimal  discretization  policy  for  a  sublinearly  convergent 
optimization  algorithm  as  well  as  its  corresponding  rate  of  convergence  of  the  total 
error  bound. 


Theorem  IV. 9.  Suppose  that  Assumption  IV.  1  holds  and  that  Asuhhn  is  a  sublinearly 
convergent  algorithm  with  degree  7  e  (0,  00)  on  {( Pnm m=m>  w%^1  ^  ant d  M  as 
in  Assumption  IV.  1.  Suppose  also  that  7fsublm  satisfies  Assumptions  IV. 2  and  IV. 3, 
and  that  {(rib,  Nb,  Mb)}bL\  * s  an  asymptotically  admissible  discretization  policy.  Then, 


log  e  (v4.sublin ,  nb ,Nb,  Mb) 

lmi  - 7 - 7 - 

b — ^00  log  0 


1 

I  _i_  H  E  ’ 

7  p  q 


(IV.57) 


where  7  is  as  in  Definition  IV.fi  p  and  q  are  as  in  Assumption  IV.  1,  and  p  and  v 
are  as  in  Assumption  IV. 3. 

Furthermore,  if  nb/b1/(',i'y/p+l"y/q+1)  — >.  0l  g  (0,  00)  and 


Nh 


p-y/p+wy/q 

bP'y/p+v~t/q+l 


pq+vp 


— t  CI2  £  (0,  00), 


(IV.58) 


as  b  — »  00,  then 


lim 

b— >00 


log  e(^4sublm,  nb,  Nb,  Mb) 


log  b 


I  _)_  M  _|_  v  ' 


7  V 


Proof.  For  any  n,  JV,MeN, 


C  K 


K 


log  e(„4“,  n,  N,M)  =  log  —  +  —  +  — 
v  y  \ni  Np  Mi 

f  (C  K  K 

“  &V  \rV  NP'  M(i 


K 


K 


=  max  <j  log  c  -  7  log  n ,  log  — ,  log  — 


Hence, 


log  e(v4sublin,  n,  N,  M) 


>  max 


log  b 

-7  log  n  +  log  C  —p  log  N  +  log  K  —q  log  M  +  log  K 


(IV.59) 


(IV.60) 


(IV.61) 


log  b  ’  log  b  ’  log  b  J  ^  ^ 

We  now  consider  three  cases,  one  for  each  of  the  three  terms  inside  the  max  function 
of  (IV.62)  when  it  is  greater  than  or  equal  to  the  other  two  terms  inside  the  max 
function. 


108 


Initially,  we  want  the  first  term  to  be  the  greater  than  or  equal  to  the  other 
two  terms.  This  will  be  true  if 


—7  log  nb  >  -p  log  Nb 

(IV.63) 

and 

—7  log  nb  >  —q  log  Mb. 

(IV.64) 

We  note  that  (IV. 63)  implies 

K  >  <1/p. 

(IV.65) 

and  (IV. 64)  implies 

>  np". 

(IV.66) 

If  we  then  use  the  bounds  on  Np  and  obtained  from  (IV. 65)  and  (IV. 66),  respec¬ 
tively,  we  have  that 

log  e(v4.sublin,  n,  N ,  M)  —7  log  nb  +  log  C 

log  b  ~  log  b 

-7  log  nb  +  log  C 

loS  (^j  +  loS  (anbNbMb) 

_ -7  log  nb  +  log  C _ 

r  \  /  pg+qMl+P]^l\ 

los  {^kw)  + log a  + log  w  ) 

_7  MV 

=  — - ; -  loenb - .  (IV.67) 

S\'ynbNbMb  )  I  logo-  I  pq+q^l+pi^l 
log  nb  '  log  nb  Pq 

We  now  consider  the  case  where  the  second  term  inside  the  max  function  of 
(IV. 62)  is  greater  than  or  equal  to  the  other  two  terms.  This  will  be  true  if 

—p  log  Nb  >  —7  log  rift,  (IV.68) 

and 

—p  log  Nh  >  —  glogMfe.  (IV.69) 

We  note  that  (IV.68)  implies 

nb  >  NPJ\  (IV. 70) 
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and  (IV.69)  implies 


Ml  >  Npv/q.  (IV.  71) 

If  we  then  use  the  bounds  on  n b  and  Ml  obtained  from  (IV. 70)  and  (IV. 71),  respec¬ 
tively,  we  have  that 

—p  log  Nb  +  log  K 


log  e(Asubhn,n,N,M) 
log  b 


> 


log  b 

-p  log  Nb  +  log  K 


> 


log  (c rnbNZM»b  )  +  bg  K. 

-p  log  Nb  +  log  K 


log  {anJzMz)  +  log*  +  ^g 


pq+qU'Y+pis'j 
<11 


~P 


log  K 

log  At, 


log 


™bNbMb 


(IV.  72) 


log  At, 


I  logo-  pq+qp'y+pu'y 

'  log  Nb  q~i 


Finally,  we  consider  the  case  where  the  third  term  inside  the  max  function  of 
(IV. 62)  is  greater  than  or  equal  to  the  other  two  terms.  This  will  be  true  if 


—q  log  Mb  >  —7  log  nb, 

(IV.  73) 

and 

—q  log  Mb  >  p  log  Nb. 

(IV.  74) 

We  note  that  (IV. 73)  implies 

nb  >  MqJ\ 

(IV.  75) 

and  (IV.  74)  implies 

Nl  >  Mq*/v. 

(IV.  76) 
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If  we  then  use  the  bounds  on  nb  and  N£  obtained  from  (IV.75)  and  (IV. 76),  respec¬ 
tively,  we  have  that 


log  e(^4sublin,  n,  N,  M) 
log  b 


> 


> 


—q  log  Mb  +  log  K 
log  b 

-q  log  Mb  +  log  K 

loS  +  loS 

-q  log  Mb  +  log  K 


lo§  +logff  +  log  ( 


pq+qp'y+pw-y 

M, 


-q  + 


log  A' 
logM6 


log 


mbKMh) 

log  Mb 


I  logo-  I 
"r  log  Mb  ^ 


pq+qp.'y+pv'y 

PI 


(IV.  77) 


We  now  consider 

log  e(»Asublm,  nbl  Nbl  Mb) 
log  b 

as  b  — *  oo.  Since  anbN{f M£ /b  — >•  1,  as  b  — >  oo,  we  consider  two  cases:  one  where  at 
least  one  of  the  parameters  nb,  Nb,  or  Mb  does  not  increase  without  bound  as  b  — *  oo, 
and  another  where  nb  — >  oo,  Nb  — >  oo  and  Mb  — >  oo,  as  b  — >  oo.  For  the  first  case,  at 
least  one  of  the  parameters  nb ,  Nb,  or  Mb  remains  finite  as  b  — »  oo.  Then  based  on 
(IV. 60),  e(Asuhhn,nb,Nb,Mb)  e  (o,  limsup^^  &  +  jtf  +  ttv\  and  we  have  that 


lim  inf 

b—>  oo 


log  e(^4sublm,  nb,  Nb,  Mb) 
log  b 


>  0  > 


I  I  P  V 

7  '  p  q 


(IV.  78) 


For  the  second  case,  where  all  three  of  the  parameters  nb,  Nbj  and  Mb  increase  without 
bound  as  b  — *  oo,  we  have  that 

log  e(*4sublin,  nb,  Nb,  Mb) 


lim  inf 

b— >oo 


log  b 


>  min  <  lim  inf 

b— >oo 


-ry  _J_  loV 

'  log  nb 


a nbNbMb  J  I  logo-  I  pq+qp'1+pvy 

log  nb  log  nb  pq 
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lim  inf 

b— >oo 


—p  + 


log  K 
log  At, 


log 


”-bKMk 


,  lim  inf 

b— >oo 


-q  + 


log  A' 
log  Mb 


log  At, 


I  logo-  pq+qm+PVl 

'  log  Nb  <77 


log 


anbNb  Mb  )  |  log  a  ,  pq+q/-H+pwy 

log  Mb  '  log  Mb  P7 


=  mm 


1  1 

l 

i  1 

i  wr 

I  i  s  _L  y 
i  p  q 

I  +  ( 

7  p  q  ) 

I  P  _I_  V 

7  p  q 


(IV.  79) 


Which  completes  the  proof  for  the  first  part  of  the  theorem. 

Next,  let  {{;nbl  Nbl  Mb)}’^=1  be  an  asymptotically  admissible  discretization  pol¬ 
icy  satisfying  n6/&1/hi7/p+''7/q+d  0l  g  (o,  oo)  and 

Nb 


p.-y/p+v'y/q  ^  pq+vp 
b  ! P+vy  / q  + 1 


— ^  Q'2  €  (0,  oo), 


as  6  — »  oo.  For  notational  simplification  we  define 


£?i  = 


Cb 


L  +  V +1 


n7 


So  = 


K 

r  P7  ^7  -i 

p  q 

M7  i  vy  _i_i 

6  p  +  <? 

pq+vp 

a 

_ 

ATP 


and 


S,  = 


P7 

P 


zz'y 

~9~ 


CT 


-pq 

pq+vp 


Then,  algebraic  manipulation  of  (IV. 56)  gives 
e(Asuhhl\nb,Nb,Mb) 

=  S16^+%+1  +S2S3  + 


A' 


b  \i/u 


\  crnN V  ) 


=  IP, 


D  ~7 

1  ,  A7  _l  A7 

- h  p  q 

s3 


+  s2  + 


/  b  \i/v  Mq 

\  crnN^  ) 

i^Y/U  Ka^n^N^ 


B3Mq 


bq/v 


(IV.80) 


(IV.81) 


(IV.82) 


(IV.83) 


(IV.84) 
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Algebraic  manipulation  can  be  used  to  show  that 


M7  ■  v'l 


(IV.85) 


M7_i_^7  o  |  U1 

q/v  p  ~  q _ pq  p  q _ pq 

b  “+ V+1  b  “+7+1  b-q/vb  “+7+1  m+vv  =b°  =  1.  (IV.86) 

Then,  since  anbN^M^/b  — »  1,  the  sum  of  terms  in  brackets  in  (IV. 84),  with  n,  N, 
and  M  replaced  by  n&,  Nb,  and  Mb,  respectively,  tends  to  a  constant  as  b  — »  oo.  The 
conclusion  for  the  second  part  of  the  theorem  then  follows  from  (IV. 84)  after  taking 
logarithms,  dividing  by  log  b,  and  taking  limits.  □ 

Based  on  Theorem  IV. 9,  the  rate  of  convergence  of  the  total  error  bound  for  a 
sublinearly  convergent  optimization  algorithm  is  worse  than  that  which  is  achievable 
by  finite,  superlinear,  and  linear  algorithms  (see  Theorems  IV. 4,  IV. 6,  and  IV. 8), 
even  when  the  optimal  discretization  policy  given  in  the  second  part  of  the  theorem 
is  used.  We  also  note  that  if  7  tends  to  infinity,  then  the  rate  for  the  sublinear  case, 
when  the  optimal  discretization  policy  is  used,  tends  to  that  of  the  finite,  superlinear, 
and  linear  cases  as  it  should. 


D.  CONCLUSIONS 

In  this  chapter,  we  consider  the  rate  of  convergence  of  a  bound  on  the  error 
between  the  objective  function  evaluated  at  iterates  generated  from  the  discretized 
problems  and  the  optimal  value  of  the  original  problem  as  a  computational  budget 
b  tends  to  infinity.  We  see  that  in  the  case  of  superlinear  and  linear  optimization 
algorithms,  the  best  possible  rate  of  convergence  is  ^/(m/p+V?^  where  /i  and  v  are 
positive  parameters  related  to  the  computational  work  per  iteration  in  the  optimiza¬ 
tion  algorithms,  and  p  and  q  are  positive  parameters  related  to  the  error  in  the 
numerical  methods  used  to  construct  the  finite-dimensional  problems.  We  identify 
specific  optimal  discretization  policies  for  both  the  superlinear  and  linear  cases  that 
achieve  this  best  possible  rate.  It  is  impossible  to  improve  upon  this  rate  due  to 
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the  fact  that  there  is  always  some  level  of  discretization  error.  If  other  discretization 
policies  are  utilized,  it  is  possible  that  they  will  result  in  substantially  slower  rates  of 
convergence.  For  sublinear  optimization  algorithms,  with  optimization  error  bounded 
by  C/n 7,  C  >  0,  7  >  0,  after  n  iterations,  the  best  possible  rate  of  convergence  is 
£,-1/(1/7+/Vp+('/<j).  This  rate  is  slower  than  what  can  be  achieved  with  either  a  super- 
linear  or  linear  optimization  algorithm.  The  optimal  discretization  policy  specified 
in  Section  IV. C. 4  achieves  the  rate  &_1/(1/t'+Vp+Vi ?).  Table  2  gives  a  summary  of  the 
results  for  the  different  optimization  algorithms  we  consider  in  this  chapter. 


Table  2.  Comparison  for  optimization  algorithms. 


Optimization  Algorithm 

Asymptotic  rate  of  decay  of  error  bound 

Finite 

-1 

fofl/p+v/q 

Superlinear 

-1 

frp/p+v/q  (with  optimal  policy) 

Linear 

b^/p+v/q  (with  optimal  policy) 

Sublinear 

-1 

bi/'i+p/p+v/q  (with  optimal  policy) 

The  results  from  this  chapter  provide  insight  regarding  the  type  of  numeri¬ 
cal  method  used  to  solve  the  differential  equations  when  implementing  a  discretiza¬ 
tion  algorithm  to  solve  one  of  the  generalized  optimal  control  problems  discussed  in 
Chapter  III.  If  a  linear  or  super  linear  optimization  algorithm  A  is  used  to  solve  the 
finite-dimensional  optimal  control  problems,  with  Euler’s  method  used  to  numerically 
approximate  the  solution  of  the  differential  equations  and  Simpson’s  rule  used  to  nu¬ 
merically  approximate  the  spatial  integration,  then  p  —  1  and  q  =  4  and  we  have 
e(A,n,  N,  M)  =  A^M(A)  +  ^  +  jji-  If  we  assume  that  Assumption  IV.3  holds  for 
the  reasons  discussed  in  Section  IV.B  with  p  —  1  and  u  —  2,  then  the  best  possible 
rate  of  convergence  is  b~ 2/3. 

If  a  Runge-Kutta  algorithm  is  used  instead  of  Euler’s  method  to  numerically 
solve  the  differential  equations,  it  is  clear  from  an  analysis  of  Runge-Kutta  algorithms 
(see,  for  example,  Nagle  &  Saff,  1989  pp.  133-134)  that  although  additional  function 
and  gradient  evaluations  are  required,  Assumption  IV.3  could  still  hold  with  p  —  1 
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and  v  —  2.  If  a  second-order  Runge-Kutta  method  is  used  instead  of  Euler’s  method 
to  numerically  solve  the  differential  equations,  then  p  —  2  and  q  =  4  and  we  have 
e(A,  n,  N,  M)  =  ArfKiM(A)  +  jpz  +  jfi-  The  resulting  best  possible  rate  of  convergence 
is  then  b _1.  If  a  fourth-order  Runge-Kutta  method  is  used  instead  of  Euler’s  method 
to  numerically  solve  the  differential  equations,  then  p  —  4  and  q  —  4  and  we  have 
e(A,n,  N,  M)  =  A^M(A)  +  yu  +  jjfj-  The  best  possible  rate  of  convergence  is  then 
6-4/3. 

In  order  to  establish  an  upper  bound  for  the  type  of  numerical  method  used 
to  solve  the  differential  equations,  we  let  p  — *  oo.  Then  we  have  e(A,n,  N,  M)  = 
AjyM(>/4)  +  -^4.  If  we  assume  that  Assumption  IV.3  still  holds  with  p  =  1  and  u  —  2, 
then  the  best  possible  rate  of  convergence  is  b~'2.  The  analysis  of  this  chapter  therefore 
indicates  that  higher-order  methods  such  as  Runge-Kutta  or  pseudo-spectral  may  lead 
to  better  overall  performance.  However,  this  potential  improvement  in  performance 
comes  at  the  cost  of  additional  complications  in  both  the  theory  and  implementation. 

We  can  also  establish  an  upper  bound  for  the  type  of  numerical  integration 
method  used  to  evaluate  the  spatial  integration  if  we  let  q  — *  00.  Then  we  have 
e(A,n,  N,  M)  =  Ar^M(A)  +  yy.  If  we  assume  that  Assumption  IV.3  still  holds  with 
p  =  1  and  v  —  2,  then  the  best  possible  rate  of  convergence  is  b~p.  The  value  of  p 
depends  on  the  choice  of  numerical  method  used  to  solve  the  differential  equations, 
with  p  =  1  for  Euler’s  method,  p  —  2  for  a  second-order  Runge-Kutta  method,  and 
p  —  4  for  a  fourth-order  Runge-Kutta  method.  Table  3  gives  a  summary  of  the 
results  for  the  different  numerical  methods  used  to  solve  the  differential  equations 
and  evaluate  the  spatial  integration. 

The  last  row  in  Table  3  gives  the  asymptotic  rate  of  decay  of  the  error  bound 
assuming  “Ideal”  methods  are  used  to  solve  the  differential  equations  as  well  as  eval¬ 
uate  the  spatial  integration.  The  term  “Ideal”  method  means  we  let  p  — >  00  and 
q  — »  00.  Then  we  have  e(A,  n,  N,  M)  =  AnNM(A).  There  is  no  benefit  associated  with 
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increasing  N  or  M,  so  b  —  n  and  the  resulting  asymptotic  rates  are  c 76  and  cb  for 
superlinear  and  linear  optimization  algorithms,  respectively. 


Table  3.  Comparison  for  numerical  methods  used  to  solve  differential  equations  and 
evaluate  the  spatial  integration.  The  optimization  algorithm  can  be  finitely,  super- 
linear  ly,  or  linearly  convergent.  The  last  row  in  the  table  gives  the  asymptotic  rate 
of  decay  of  the  error  bound  assuming  “Ideal”  methods  are  used  to  solve  the  differ¬ 
ential  equations  as  well  as  evaluate  the  spatial  integration.  The  rates  given  are  for  a 
superlinear  optimization  algorithm  with  order  7  G  (1,  00)  and  c  G  (0, 1),  and  a  linear 
optimization  algorithm  with  rate  constant  c  G  (0, 1). 


Numerical  Method  for 

Numerical  Method  for 

Asymptotic  rate  of 

Differential  Equations 

Spatial  Integration 

decay  of  error  bound 

Euler 

Simpson 

6  A 

2nd  Order  Runge-Kutta 

Simpson 

b~l 

4th  Order  Runge-Kutta 

Simpson 

b ir 

“Ideal”  p  — >  00 

Simpson 

b-1 

Euler 

“Ideal”  q  — >  00 

b-1 

2nd  Order  Runge-Kutta 

“Ideal”  q  — >  00 

b~2 

4th  Order  Runge-Kutta 

“Ideal”  q  — >  00 

b~4 

“Ideal”  p  — >■  00 

“Ideal”  q  — >  00 

c1  or  c 

We  leave  verification  of  the  theoretical  rate  of  convergence  results  developed 
in  this  chapter  via  numerical  examples  for  future  work.  We  note  that  this  verification 
will  be  challenging  clue  to  the  fact  that  it  will  be  difficult  to  determine  what  the 
optimal  value  of  the  objective  function  should  be  for  the  problems  under  considera¬ 
tion.  The  theoretical  results  of  this  chapter  are  still  useful,  however,  as  they  provide 
insight  regarding  the  choice  of  numerical  methods  for  approximately  solving  the  dif¬ 
ferential  equations  and  approximating  the  spatial  integrals  as  well  as  the  optimization 
algorithms  that  can  be  utilized  to  implement  a  discretization  algorithm  that  solves  a 
generalized  optimal  control  problem. 
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V.  ALGORITHMS  AND  NUMERICAL 

RESULTS 

This  chapter  bridges  the  gap  between  the  theory  developed  in  Chapter  III  and 
implementable  algorithms  that  can  be  used  to  solve  the  problems  ( GTP ),  ( GTPC ), 
(< GTPe ),  ( GTPc’e ),  (JTPe),  (JTPc,e),  ( ITPp ),  and  (ITPC’P).  Again,  for  the  sake 
of  brevity  we  omit  explicit  treatment  of  the  problems  ( GTPe )  and  ( GTPc,e )  in  this 
chapter.  The  results  developed  in  this  chapter  for  ( ITPe )  and  (JTPc,e)  can  again 
be  trivially  extended  to  include  (GTPe)  and  (GTPc,e),  respectively,  with  a1  replaced 
by  a  and  0Z(-)  replaced  by  0(-).  In  Section  V.A,  we  discuss  problems  defined  on  a 
real  Euclidean  space  of  coefficients  needed  to  construct  implementable  algorithms, 
followed  by  a  statement  of  the  algorithms  and  their  proofs  of  convergence,  where 
appropriate.  Then,  in  Section  V.B.l,  we  use  one  of  these  algorithms,  with  a  fixed 
discretization  scheme,  to  solve  an  instance  of  ( GTPC ).  This  solution  is  then  used  to 
derive  operational  insights  on  how  to  better  defend  a  HVU  against  an  attack  from  a 
collection  of  small  boat  adversaries.  We  also  give  examples  of  numerical  solutions  to 
instances  of  (ITPc,e)  and  ( ITPC,P )  using  fixed  discretization  schemes.  In  an  effort  to 
develop  faster  solution  methods,  in  Section  V.B.2  we  use  an  algorithm  with  an  adap¬ 
tive  precision-adjustment  scheme  to  solve  ( GTPC )  and  compare  the  results  acheived  to 
those  obtained  using  different  fixed  discretization  schemes.  Finally,  in  Section  V.B.3, 
we  compare  three  heuristic  methods  designed  for  use  onboard  unmanned  systems  to 
provide  solutions  to  ( GTPC )  in  real  time. 

A.  IMPLEMENTABLE  ALGORITHMS 

The  approximating  problems  ( GTPNM ),  (GTF%M)7  (ITPfrM),  (ITP^eM), 
(ITPxM),  and  (ITP^pM)  in  Chapter  III  were  all  defined  using  the  function  space  Hn . 
We  would  like  to  use  existing  nonlinear  programming  algorithms  to  solve  these  ap¬ 
proximating  generalized  optimal  control  problems.  The  issue  with  using  these  existing 
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algorithms  is  that  they  are  defined  only  on  real  Euclidean  spaces.  For  this  reason, 
we  need  to  define  equivalent  problem  formulations  in  a  corresponding  real  Euclidean 
space  of  coefficients  that  we  can  use  to  calculate  numerical  solutions.  Equivalence  is 
based  on  correspondence  between  solutions  as  in  sections  4.3  and  5.6  of  Polak  (1997). 
We  adopt  the  notation  used  in  sections  4.3  and  5.6  of  Polak  (1997)  to  define  these 
problems,  and  mirror  the  development  found  on  pages  722-723  of  Polak  (1997)  in  this 
section. 

If  we  let  e3  denote  the  jth  unit  vector  in  Mm,  then  the  functions  e3i Tjv,k(')j 
j  =  1  =  0 —  1,  with  Hn,k  defined  as  in  (111.54),  form  an  orthonormal 

basis  for  the  subspace  L n  defined  in  (111.55).  This  means  that  Hn  is  in  one-to-one 
correspondence  with  the  real  Euclidean  space 

Hn  =  IT  x  RmN.  (V.l) 


The  elements  of  HN  are  fj  =  (£,  u),  with  £  G  Mn  and  u  =  (u0,  Wi,  •••,  un-i)  €  Mmxlmx 
...  x  Mm.  Because  Hn  is  in  one-to-one  correspondence  with  Hn,  any  rj  =  (£,  u)  G  Hn, 
with  u(-)  =  T,^uKnN,K{-),  corresponds  to  a  unique  fj  =  (£,h)  G  Hn,  with  u  split 
into  N,  m-dimensional  blocks  u  =  (ho,  hi,  ...,un-i)-  Then  for  any  N  G  A f,  we  dehne 
the  linear,  invertible  maps  Wn  '■  Hn  —f  Hn  by 


N—l 


WN  U.E  'Ujk,'^N,k, (*)  J  —  (£5  ^1?  •••?  UN—  l))  • 


For  any  77,7/  G  Hn  and  77  =  Wn{ 77),  77'  =  Wn( 77'), 

(v, v)h2  =  {£> O  +  [  («(*), u'{t)) dt 


N-l 


(£,£')  +  ('/NuK,\/Nu'h 


K= 0 


N-l 


(&  o  +  u «)  -  fa  , 


Av— 0 


(V.2) 


(V.3) 


where  (•,  •)  denotes  the  Euclidean  inner  product  on  and  (•,  -)g  denotes  the  Eu¬ 
clidean  inner  product  on  Hn-  Which  means  for  any  rj  G  Hn  and  fj  =  Wn (v) , 

IMlk  =  (h,  hU  =  IlhllL  =  (h,  v)hn  •  (v-4) 
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This  shows  that  Wn ( • )  is  a  norm-preserving,  or  isometric,  map  whose  adjoint  is 
1T’^1(-),  which  means  given  any  q  G  Hn  and  q  G  Hn, 

(v>Wn(v))hn  ~  (^n1  (v)  iV)  Hn  ■  (V.5) 

Given  an  q  =  (£ ,u )  G  Hn,  if  we  let  77  =  U41  (77 ) ,  then  (III. 60)  defines  the 

I  _  >|  v 

sequence  of  vectors  4  —  1  %n  (j)  \  in  for  all  k  —  1,  2, ...,  K,  by  the  recursion 

l  J  j= 0 

4*  «J  +  1)/ TV)  -  (i/JV)  =  Vfc‘  (4*  074.  VNu*(j/N))  , 

j  6  {0.1,..,iV-l},4(0)={.  (V.6) 

Similarly,  (III. 61)  and  (III.  181)  define  the  sequence  of  vectors  zjv  =  {4(i;«)}L) G  M 
_  a  r_-i 

and  zlN=  \z]}  (j',  cd)  >  G  M  by  the  recursions 
l  J  j= 0 

I<  L 

Zn((J  +  l)/N',a)  ~  zvN(j/N-,a)  =  vEE--w(4‘  (j/fV),^  (j/iV;a)), 

v  fc=i  ;=i 

J  e  {0,1,...,1V-I},4(0;a)  =  0,  (V.7) 


and 


4*  (( 3  +  i)/^V; a1)  -  4'  (i/4  «')  =  rk'1  {xNkU/N),yl  (i/4 al))  > 

v  fc=i 


J  e  {0,1,...,1V-1},4(0;^)=0,  (V.8) 


respectively. 

Using  the  discretized  “information  state”  given  by  the  recursion  (V.7),  we 
define  the  approximate  objective  functions  Jnm  '■  HN  — »  M  for  any  q  G  74  and 
IV  G  A/”  by 

/tvm('7)  -  Im  (exp  [-44 ')]  </>(•))  »  (V.9) 

and  the  corresponding  discrete  generalized  optimal  control  problems  by 

(■ GTPnm )  min  Jnm{j]),  (V.10) 

v&hn 
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and 


(< CrTPcNM )  min  fNM(fj),  (V.ll) 

CjJv 

where 


Hcjv  ~ 


x  {u  =  (u0,Ui,  Wjv-l) 


nmJV 


Un  G 


U,j  G  {0, 1, ...,  IV  —  1}}  ,  (V.12) 


with  [/  as  in  (III. 9). 

Similarly,  using  the  discretized  “information  state”  given  by  the  recursion 
(V.8),  we  define  the  approximate  objective  functions  i^NM  :  Hn  — >•  M  and  V’atm  : 
Pat  — *  M.  for  any  fj  G  h/jv  and  TV  G  A/”  by 

L 


(exp  </>z(-)), 


(V.13) 


z=i 


and 


$ nm(v )  -  n  Im  (exp  ■ 


«=i 


(V.  14) 


We  define  the  corresponding  discrete  generalized  optimal  control  problems  by 


(ITPlnm) 

min 

rj£HN 

(V.  15) 

(ITP%eM) 

min 

fjeilc,N 

(V.16) 

(ITPpnm) 

min 

fj£HN 

(V.17) 

(ITP^npm) 

min 

>teHCjAr 

i’NM (v)- 

(V.18) 

Based  on  Exercise  4.3.7  in  Polak 

(1997),  the  problems  (GTPnm, 

).  (gtp;m), 

(. ITPeNM ),  (ITPCj^m),  ( ITPpnm ),  and  (ITPC^M)  are  equivalent  to  the  problems 
(GTPnm),  ( GTPCNM ),  (JTP£m),  (/TP^m),  (ITPpnm),  and  ( ITPC/M ),  respectively, 
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in  the  sense  that  they  are  related  by  a  nonsingular  linear  transformation,  and  that  at 
corresponding  points  the  values  of  their  corresponding  optimality  functions  are  the 
same. 

We  now  state  generic  algorithm  models  based  on  Master  Algorithm  Model 
3.3.12  from  Polak  (1997)  that  can  be  used  to  solve  the  problems  ( GTP ),  (GTPC), 
(ITPe),  ( ITPc,e ),  ( ITPP ),  and  ( ITPC,P ).  Our  algorithms  define  the  “outer”  it¬ 
erations,  while  the  “inner”  iterations  are  defined  by  user  supplied  iteration  maps 
A((P),fj),  where  A  denotes  one  iteration  of  a  nonlinear  programming  solver  applied 
to  problem  (P)  starting  from  fj.  We  begin  with  the  unconstrained  problems  and  state 
the  following  algorithm,  which  is  based  on  a  fixed  discretization  scheme. 

Algorithm  V.l.  Approximately  solves  (GTP). 

Data:  N0  £  Af ,  M0  G  N3  x  N35, 770  £  H^. 

Step  0.  Set  N  =  N0,  M  =  M0,  and  fjo  =  Wn(vo)- 

Step  1.  Generate  {fji}°Z0  using  fji+1  G  A((GTPNM),fji).  □ 

Algorithm  V.l  also  approximately  solves  ( ITPe )  and  (ITPP),  with  (GTP^m)  re- 
placed  by  ( ITPeNM )  and  (ITPpnm),  respectively.  For  the  constrained  problems,  we 
have  the  following  algorithm. 

Algorithm  V.2.  Approximately  solves  (GTPC). 

Data:  V0  G  A/",  M0  G  N3  x  N3,  r]0  G  H,,jAr0. 

Step  0.  Set  N  =  N0,  M  =  M0 ,  and  fjo  =  Wn(7]0)- 

Step  1.  Generate  {^}“0  using  fji+1  G  A((GTPCNM),  fji).  □ 

Algorithm  V.2  also  approximately  solves  (ITPc'e)  and  (ITPC,P),  with  (GTPCNM)  re¬ 
placed  by  (ITPC^M)  and  (IT PCj(fM),  respectively. 

In  order  to  improve  the  run-time  performance  of  Algorithms  V.l  and  V.2,  we 
also  develop  algorithms  based  on  an  adaptive  precision-adjustment  scheme.  Algo¬ 
rithms  based  on  adaptive  precision-adjustment  keep  the  discretization  parameters, 
N  and  M,  small  when  the  algorithm  starts  and  is  “far”  from  the  optimal  solution. 

5Recall  that  N3  =  {rn  £  2N  +  1| m  >  3},  as  defined  in  Proposition  III. 26. 
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We  use  the  optimality  function  to  determine  how  “far”  the  current  iterate  is  from 
the  optimal  solution.  The  values  of  N  and  M  are  then  increased  according  to  suc¬ 
cessor  functions  as  the  algorithm  gets  closer  to  the  optimal  solution.  One  advantage 
these  algorithms  have  over  fixed  precision  algorithms  is  that  the  computational  work 
remains  relatively  low  until  the  algorithm  gets  close  to  the  optimal  solution.  An¬ 
other  advantage  is  that  solutions  obtained  using  low  levels  of  discretization  are  used 
to  “warm  start”  the  algorithm  at  higher  discretization  levels.  Algorithms  based  on 
adaptive  precision- adjustment  fit  nicely  within  the  framework  of  consistent  approxi¬ 
mations  because  we  know  that  as  the  values  of  N  and  M  tend  to  infinity,  the  solution 
of  the  approximating  problem  converges  to  the  solution  of  the  original  problem.  These 
algorithms  use  successor  function  k  :  Af  — >■  Af  defined  by 

k(N)  G  {N'  G  Af\N'  >  N}  .  (V.19) 

It  is  clear  from  (V.19)  that  k(N)  is  an  integer  larger  than  N  G  Af.  We  begin  with 
the  unconstrained  problems  and  state  the  following  algorithm,  which  is  based  on  an 
adaptive  precision-adjustment  scheme. 

Algorithm  V.3.  Solves  ( GTP ). 

Data:  N0  G  Af,  M(N)  :AgN3xN3,  rj0  G  H^. 

Parameter:  /A,  An  As  >  0. 

Step  0.  Set  i  —  0,  N  —  N0,  M  =  M(N0),  and  fji  =  Wjy(r]i). 

Step  1.  Compute  an  fji+1  G  A((GTPNM^N)),fji). 

Step  2.  If  6(W^(fji+1))  >  -(A/A  +  /32/(M1(A))4  +  /33/(M2(A))4),  set  rj*N  = 
W^1  ,  and  replace  N  by  n(N). 

Step  3.  Replace  i  by  i  +  1,  and  go  to  Step  1.  □ 

Algorithm  V.3  also  solves  ( ITPe )  and  ( ITPP ),  with  (GTP nm(n))  replaced  by 

(ITPeNM(N^)  and  (ITPpn MrN\),  and  d(-)  replaced  by  6e(-)  and  0P(-),  respectively.  Then 

the  following  theorem  is  a  direct  consequence  of  Theorem  III. 29. 

Theorem  V.l.  If  the  sequences  and  {r/^}  are  constructed  by  Algorithm  V.3, 

then 
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(i)  if  the  sequence  { rj is  finite,  then  the  sequence  has  no  accumulation 

points;  and 

(ii)  if  the  sequence  {77^}  is  infinite,  then  every  accumulation  point  77*  G  H°  of 

{^}-0,  satisfies  6{r f)  =  0.  □ 

When  Algorithm  V.3  is  used  to  solve  ( ITPe )  and  ( ITPP ),  a  result  similar  to  Theorem 
V.l  with  #(•)  replaced  by  9e(-)  and  9P(-),  respectively,  follows  as  a  direct  consequence 
of  Theorem  III. 52. 

For  the  constrained  problems,  we  have  the  following  algorithm. 

Algorithm  V.4.  Solves  ( GTPC ). 

Data:  Nq  G  A/",  iff  (A)  :  J\f  — >  N3  x  M3, 770  £  Hcjv0. 

Parameter:  /A, /32,  fis  >  0. 

Step  0.  Set  i  =  0,  A  =  A0,  iff  =  iff(A0),  and  fy  =  Wv(?/j). 

Step  1.  Compute  an  77^+1  G  A((GTPcNM^),fji). 

Step  2.  If  >  -(A/A  +  ^M^Aj^  +  ^AMalA))4),  set  rj*N  = 

W^itii+i),  and  replace  A  by  k(N). 

Step  3.  Replace  i  by  i  +  1,  and  go  to  Step  1.  □ 

Algorithm  V.4  also  solves  (iTPc,e)  and  ( ITPC,P ),  with  ( GTPCNM replaced  by 
{ITPcf^M^)  and  {ITPcf^M^),  and  dc(-)  replaced  by  6|c,e(-)  and  0C,P(-),  respectively. 
Then  the  following  theorem  is  a  direct  consequence  of  Theorem  III. 30. 

Theorem  V.2.  If  the  sequences  {fj and  {77^}  are  constructed  by  Algorithm  V.f, 
then 

(i)  if  the  sequence  {77^}  is  finite,  then  the  sequence  {'Iy}fiL{)  has  no  accumulation 
points;  and 

(ii)  if  the  sequence  {rfN}  is  infinite,  then  every  accumulation  point  77*  G  Hc  of 

{Atv } i=o ’  satisfies  6C( rf)  =  0.  □ 

When  Algorithm  V.4  is  used  to  solve  ( ITPc,e )  and  ( ITPC,P ),  a  result  similar  to  The¬ 
orem  V.2  with  9C(-)  replaced  by  9c,e(-)  and  6C,P(-),  respectively,  follows  as  a  direct 
consequence  of  Theorem  III. 53. 
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Theorems  V.l  and  V.2  are  the  end  result  of  the  theory  we  develop  in  Chapter 
III.  Theorems  V.l  and  V.2  establish  the  fact  that  the  discretization  schemes  we 
develop  in  Chapter  III  result  in  implementable  algorithms  that  are  guaranteed  to 
converge  to  stationary  points  of  the  original  problems. 

B.  NUMERICAL  RESULTS 

1.  Fixed  Discretization  Schemes 

In  this  section  we  present  numerical  results  based  on  the  situational  description 
given  in  Chapter  II.  We  start  by  solving  three  problem  instances  given  in  Table  4 
using  Algorithm  V.2,  with  parameter  values  given  in  Table  5.  The  column  headings 
K  and  L  in  Table  4  denote  the  number  of  searchers  and  targets,  respectively.  The 
column  heading  fjo  in  Table  5  represents  the  initial  “guess”  for  the  algorithm.  The 
first  three  elements  of  fjo  are  the  initial  headings  of  the  searchers  measured  from  the 
horizontal  axis.  The  initial  control  portion  of  fjo  is  0,  which  represents  the  zero  vector 
of  length  No-  We  note  that  we  do  not  include  the  initial  position  of  the  searchers  in 
fjo,  as  they  are  assumed  to  be  fixed  and  not  part  of  the  decision  vector.  The  initial 
positions  for  the  searchers  are  provided  in  Table  7.  The  parameter  values  given  in 
the  first  row  of  Table  5  are  used  with  Algorithm  V.2  for  ProbA  and  ProbC,  and  the 
parameter  values  given  in  the  second  row  are  used  with  Algorithm  V.2  for  ProbB. 

Table  4.  Fixed  discretization  problem  instances. 


Instance 

Problem  Class 

K 

L 

ProbA 

(< GTPC ) 

3 

10 

ProbB 

(. ITPc’e ) 

3 

2 

ProbC 

(ITPC,P) 

3 

1 

Table  5.  Algorithm  V.2  parameters. 


No 

M0 

Vo 

200 

(25,25) 

(tt/4,  tt/2,  tt/2,  0) 

320 

(25,25) 

(tt/4,  tt/2,  tt/2,  0) 
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In  ProbA,  ProbB,  and  ProbC  we  consider  the  case  of  a  HVU  operating  in  a 
two-dimensional  area  of  interest  (AOI),  measuring  70  nautical  miles  (nm)  by  70  nm. 
The  HVU  follows  a  straight  line  trajectory,  with  constant  heading  £3  and  constant 
speed  u°  from  its  initial  position  x°(0).  The  parameter  values  for  the  HVU  are  given 
in  Table  6.  While  in  transit,  the  HVU  is  under  threat  of  attack  from  L  targets.  The 
target  trajectories  are  conditioned  upon  the  random  vector,  a,  with  realization  in 
A  C  M2.  For  our  numerical  results,  the  target  trajectories  are  conditioned  upon  a 
random  starting  time  between  zero  and  one  hour,  and  a  random  starting  location  on 
either  of  the  vertical  sides  of  the  AOI.  The  numerical  solutions  we  find  for  ProbA, 
ProbB,  and  ProbC  are  all  based  on  a  receding  horizon  search,  where  we  are  planning 
for  the  next  hour. 

To  generate  numerical  solutions  for  ProbA,  ProbB,  and  ProbC,  we  implement 
and  run  Algorithm  V.2  in  MATLAB  7.11.0  (version  2010b)  (see  Math  Works,  2011) 
on  a  3.46  GHz  PC  with  two  quad-core  processors,  using  Windows  7  Pro  64-bit,  with 
24  GB  of  RAM.  We  use  the  SQP  algorithm  in  TOMLAB  SNOPT  solver  (see  Gill 
et  ah,  2007),  with  the  default  major  and  minor  optimality  tolerance  as  the  stopping 
criteria. 

In  ProbA,  the  HVU  is  under  threat  of  attack  from  L  =  10  targets.  We  assume 
that  the  distributions  for  the  targets’  starting  time  and  starting  location  are  uni¬ 
form  and  independent.  The  range  of  the  uniform  distribution  for  starting  location  is 
[0, 140],  as  we  combine  both  sides  of  the  AOI  into  a  single  segment.  The  range  of  the 
uniform  distribution  for  the  starting  time  is  [0,1].  The  targets  follow  deterministic 
trajectories,  {y\t;a)  :  0  <  t  <  l},  given  a,  where  yl(t]a )  =  a),  yl2(t;  a))T  G  M2 

is  the  position  of  the  Ith  target  at  time  t. 

The  target  trajectories  are  generated  by  solving  the  optimal  control  problem 
described  in  Section  II. C  with  an  additional  constraint  for  each  target.  The  additional 
constraint  requires  the  targets  to  hit  the  HVU  at  a  90  degree  angle  of  incidence  at  the 
final  time,  tf.  The  parameter  values  used  to  formulate  this  optimal  control  problem 
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are  given  in  Table  6.  The  starting  position  for  the  Ith  target,  yl0(a),  is  on  one  of  the 
vertical  sides  of  the  AOI,  and  is  determined  based  on  the  level  of  spatial  discretization. 
We  assume  that  the  separation  in  starting  position  between  members  of  the  swarm  is 
1.5  nm.  The  target  trajectories  are  truncated  because  they  are  generated  by  solving 

Table  6.  Target  and  HVU  parameter  values.  The  target  parameter  values  are  the 
same  for  all  targets  l  =  1,  2, ...,  L. 


vlo 

vf 

Vlmin 

^ max 

yljtor 

x°(0) 

x3 

v° 

15  kts 

19  kts 

1  kt 

35  kts 

250  (35,0)  (nm,nm) 

tt/2 

25  kts 

a  time-optimal  control  problem,  and  our  search  problem  uses  a  receding  horizon 
approach  based  on  the  planning  horizon  [0,1].  An  example  of  this  truncation  is 
illustrated  in  Figure  3,  where  the  trajectory  of  the  HVU  is  indicated  in  blue,  the  first 
component  of  the  vector  M  gives  the  number  of  starting  locations  for  the  swarm, 
and  the  second  component  of  the  vector  M  gives  the  number  of  starting  times  for 
the  swarm.  The  graph  on  the  top  left  of  Figure  3  shows  the  trajectories  for  a  single 
swarm  consisting  of  L  =  10  targets.  The  middle  graph  in  the  top  row  of  Figure  3 
indicates  the  different  trajectories  that  the  swarm  would  follow  if  it  started  from  five 
specific  locations.  The  trajectories  for  the  different  starting  locations  are  shown  in 
alternating  colors,  red  and  then  cyan.  The  graph  on  the  top  right  of  Figure  3  shows 
the  different  trajectories  that  the  swarm  would  follow  if  it  started  from  five  specific 
locations  and  two  specific  starting  times.  The  trajectories  for  the  different  starting 
locations  are  shown  in  alternating  colors,  red  and  then  cyan,  for  the  first  starting  time. 
The  trajectories  for  the  different  starting  locations  are  shown  in  alternating  colors, 
green  and  then  purple,  for  the  second  starting  time.  The  graphs  on  the  bottom  row  of 
Figure  3  are  the  same  as  their  counterparts  in  the  top  row,  except  they  are  truncated 
to  show  how  far  the  targets  (and  HVU)  can  progress  on  their  desired  trajectories 
during  the  1  hour  planning  horizon. 

In  an  effort  to  detect  the  targets,  the  HVU  has  an  escort  consisting  of  K  =  3 
searchers.  The  goal  of  the  searchers  is  to  minimize  the  probability  of  failing  to  detect 
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M=(l,l) 


M=(  5,2) 


Figure  3.  Full  and  truncated  target  trajectories  for  L  =  10.  Top  Row  Left:  M  =  (1, 1) 
Full,  Top  Row  Middle:  M  =  (5, 1)  Full,  Top  Row  Right:  M  =  (5,  2)  Full.  Bottom 
Row  Left:  M  =  (1, 1)  Truncated,  Bottom  Row  Middle:  M  =  (5, 1)  Truncated, 
Bottom  Row  Right:  M  =  (5,  2)  Truncated. 

any  of  the  targets  within  the  planning  horizon  [0, 1].  For  any  t  G  [0, 1],  let  xk{t )  = 
(x\(t) ,  x^it) ,  xk(t))T  G  M3  be  the  physical  state  of  the  kth  searcher  at  time  t,  where 
x\{t)  G  M  and  xk(t)  G  M  are  the  horizontal  and  vertical  components  of  the  location 
of  the  kth  searcher,  respectively,  and  xk(t)  G  K  is  the  heading  of  the  kth  searcher 
measured  from  the  horizontal  axis.  We  assume  that  the  searchers  are  Dubins  vehicles 
traveling  at  a  constant  velocity  vk  >  0.  The  control  input  for  the  kth  searcher,  uk  G  M, 
is  the  rate  of  change  of  the  heading  (also  known  as  the  yaw  rate)  of  the  kth  searcher. 
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Then,  the  searcher  dynamics  described  in  general  terms  by  (11.22)  are  given  by 


hk(xk(t),uk(t))  4 


vk  cos  xk(t) 
vk  sin  xk(t) 
uk(t) 


\ 


,  k  =  1,2,3. 


(V.20) 


We  consider  two  different  search  platforms  for  our  numerical  results.  One  is  modeled 
after  an  SH-60  helicopter,  and  the  other  is  modeled  after  a  guided  missile  destroyer 
(DDG).  The  searcher  parameters  are  given  in  Table  7,  where  k  —  1  is  the  helicopter 
and  k  =  2,  3  are  the  destroyers.  The  initial  heading  of  the  kth  searcher  is  given  by  £ k , 
and  the  initial  position  of  the  kth  searcher  is  given  by  xfc(0).  The  yaw  rate  limit  for 
the  DDG  is  based  on  discussions  with  CDR  David  Schiffman,  USN,  a  surface  warfare 
officer  in  the  Operations  Research  Department  at  NPS  (personal  communication, 
November  2010),  and  the  yaw  rate  limit  for  the  helicopter  searcher  is  based  on  the 
limit  for  the  SH-60  helicopter  found  in  A1-H60BB-NFM-000  (SH-60  NATOPS  Flight 
Manual,  1996). 


Table  7.  Searcher  parameter  values. 


k  =  1 

k  =  2 

k  =  3 

e 

7r/4 

vr/2 

tt/2 

vk 

120  kts 

25  kts 

25  kts 

xk(0) 

(35,0)  (nm,nm) 

(25,0)  (nm,nm) 

(45,0)  (nm,nm) 

Yaw  Rate  Limit 

1885  — - 

hour 

250  t~— — ■ 

hour 

250  ^ 

hour 

We  let  rk,l(xk(t),yl(t',a))  >  0  denote  the  detection  rate  at  time  t  for  the  kth 
searcher  at  (xk(t),xk(t))T  when  the  Ith  target  is  located  at  yl(t ;  a).  For  our  numerical 
results,  we  adopt  the  Poisson  Scan  Model  (see,  e.g.,  p.  3-1  in  Washburn,  2002)  and 
set 


rk\(xk(t),xk(t)f,y\t-a))  =  Afc$[(F*  -  p((xk(t),  xk(t))T ,  y\t-  a)))/a%  (V.21) 


where  <&(•)  is  the  standard  normal  cumulative  distribution  function,  Xk  is  the  scan 
opportunity  rate,  Fk  is  a  sensor  parameter,  ak  reflects  the  variability  in  the  received 


128 


signal  strength,  and  pk((xk(t),  xk(t))T,  yl(t ;  a))  is  used  to  model  the  signal  loss,  which 
depends  on  the  distance  between  the  target  and  the  searcher;  see  for  example  Figure 
4.5  on  page  93  in  Wagner  et  ah  (1999).  The  detection  rate  functions  for  the  helicopter 
searcher,  shown  in  Figure  4,  and  the  DDG  searchers,  shown  in  Figure  5,  are  based 
on  pk((xk(t),xk(t))T,yl(t;a))  =  ak\\(xk(t),  xk(t))T  —  yl(t;  a)||2  +  bk,  with  parameter 
values  given  in  Table  8.  The  parameter  values  are  chosen  to  reflect  reasonable  sensor 
performance  based  on  discussions  with  other  naval  officers  in  the  Operations  Research 
Department  at  NPS  who  have  served  on  these  platforms  (CDR  David  Schiffman,  USN, 
and  CDR  Douglas  Burton,  USN,  personal  communication,  November  2010). 


Table  8.  Detection  rate  parameter  values. 


k  =  1 

k  =  2 

k  =  3 

xk 

1.1 

1 

1 

pk 

90 

90 

90 

ak 

0.3 

0.3 

0.3 

bk 

20 

60 

60 

ak 

100 

15 

15 

The  dimension  of  the  decision  vector,  fj(-),  in  ProbA  is  KNq+K  =  603,  because 
we  determine  a  control  input  for  each  of  the  searchers  at  every  time  step  as  well  as  the 
optimized  initial  heading  for  each  searcher.  Before  we  give  our  numerical  results,  we 
first  show  baseline  results  based  on  the  current  concept  of  operations  (CONOPS)  in 
use  by  the  U.S.  Navy.  This  CONOPS  has  the  helicopter  escort  orbiting  the  HVU  at  a 
distance  of  2.5  nm  and  the  DDG  escorts  keeping  station  on  parallel  courses  to  the  HVU 
at  a  distance  of  10  nm.  The  CONOPS  is  based  on  discussions  with  helicopter  pilots 
and  surface  warfare  officers  in  the  Operations  Research  Department  at  NPS  (LCDR 
Harrison  Schramm,  USN,  LCDR  Ron  Cappellini,  USN,  and  CDR  David  Schiffman, 
USN,  personal  communication,  February  2011).  The  resulting  trajectories  are  shown 
in  Figure  6.  In  Figure  6,  the  target  trajectories  are  plotted  in  alternating  colors  of 
red  and  cyan  to  separate  the  specific  starting  positions  of  the  swarm.  It  should  be 
noted  that  only  the  set  of  target  trajectories  based  on  a  starting  time  of  t  =  0  and  all 
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1  - 


Figure  4.  Detection  Rate  Function  for  Helo  Searcher. 

starting  locations  is  plotted  in  Figure  6.  The  additional  target  trajectories  based  on 
other  starting  times  between  [0, 1]  are  not  shown  in  the  interest  of  keeping  the  plot 
legible.  The  trajectory  for  the  HVU  is  shown  in  blue,  the  DDG  escort  trajectories  are 
shown  in  black,  and  the  helicopter  escort  trajectory  is  shown  in  green.  The  objective 
value,  which  represents  the  probability  that  all  of  the  searchers  fail  to  detect  any  of 
the  targets  in  [0, 1],  for  this  set  of  trajectories  is  0.9589.  It  should  be  noted  that  as  the 
targets  get  closer  to  the  HVU  the  probability  that  all  of  the  searchers  fail  to  detect 
any  of  the  targets  will  be  lower,  but  this  improvement  might  come  too  late  for  the 
HVU  to  effectively  defend  itself. 

The  resulting  trajectories  from  Algorithm  V.2  are  shown  in  Figure  7.  The 
colors  for  the  trajectories  are  the  same  as  those  described  above  for  Figure  6.  The 
helicopter  trajectory  is  particularly  good  at  illustrating  how  the  algorithm  gets  the 
searchers  to  areas  where  they  can  accumulate  as  much  probability  “mass”  as  possible, 
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Figure  5.  Detection  Rate  Function  for  DDG  Searcher. 

and  keeps  them  there  to  the  extent  allowed  by  the  constant  speed  and  turn  rate 
constraints.  The  objective  value  is  now  0.3954,  which  is  significantly  better  than  the 
baseline.  This  indicates  there  is  a  benefit  to  leaving  the  vicinity  of  the  HVU  and 
seeking  out  the  potential  attackers.  The  time  required  to  solve  ProbA  was  26.82 
hours,  including  1.54  hours  to  generate  the  target  trajectories. 

In  ProbB  the  HVU  is  under  threat  of  attack  from  L  —  2  targets.  The  target 
l  =  1  is  assumed  to  leave  from  the  right-hand  side  of  the  area  of  interest  and  target 
l  —  2  is  assumed  to  leave  from  the  left-hand  side  of  the  area  of  interest.  The  target 
trajectories  are  again  conditioned  upon  a  random  starting  time  between  zero  and 
one  hour,  and  a  random  starting  location  along  the  appropriate  vertical  side  of  the 
AOI.  We  assume  that  the  target’s  starting  time  and  location  are  independent  random 
variables.  We  assume  that  the  distribution  for  starting  position  for  both  targets  is 
uniform  with  range  [0,70].  We  assume  that  the  distribution  for  starting  time  for 
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Figure  6.  Trajectories  based  on  current  CONOPS.  HVU  trajectory  is  blue.  Helicopter 
trajectory  is  green.  DDG  trajectories  are  black.  Target  trajectories  alternate  red  and 
cyan. 

target  l  —  1  is  triangular,  with  the  target  being  twice  as  likely  to  leave  at  t  =  0  as 
it  is  to  leave  at  t  —  1.  We  assume  that  the  distribution  for  starting  time  for  target 
l  —  2  is  uniform  with  range  [0, 1]. 

The  target  trajectories  are  generated  in  the  same  manner  as  they  were  for 
ProbA  with  the  parameter  values  given  in  Table  9. 

Table  9.  Target  and  HVU  parameter  values.  The  target  parameter  values  are  the 
same  for  targets  l  —  1,2. 


vlo 

vf 

vLin 

^ max 

j-.l,tar 

LL 

x°(0) 

coo 

v° 

10  kts 

12  kts 

1  kt 

35  kts 

250  (35,0)  (nm,nm) 

tt/2 

25  kts 

The  HVU  again  has  an  escort  consisting  of  K  =  3  searchers.  The  goal  of  the 
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Figure  7.  Trajectories  based  on  Algorithm  V.2  on  ProbA.  HVU  trajectory  is  blue.  He¬ 
licopter  trajectory  is  green.  DDG  trajectories  are  black.  Target  trajectories  alternate 
red  and  cyan. 

searchers  is  to  maximize  the  expected  number  of  targets  detected  within  the  planning 
horizon  [0,1].  The  searcher  and  detection  rate  parameters  are  the  same  as  those  given 
in  Tables  7  and  8,  respectively. 

The  dimension  of  the  decision  vector,  fj(-),  in  ProbB  is  again  given  by  KN0  + 
K  =  963,  which  is  different  value  than  it  was  for  ProbA  due  to  the  different  parameter 
values  used  in  Algorithm  V.2.  The  resulting  trajectories  from  Algorithm  V.2  are 
shown  in  Figure  8.  The  color  codes  for  the  trajectories  are  the  same  as  described 
above  for  Figure  6,  except  that  target  l  =  1  trajectories  are  shown  in  red  and  target 
l  =  2  trajectories  are  shown  in  magenta.  Again,  we  note  that  only  the  set  of  target 
trajectories  based  on  a  starting  time  of  t  —  0  and  all  starting  locations  is  plotted 
in  Figure  8.  The  objective  value,  which  represents  the  expected  number  of  targets 
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detected  in  [0,1],  for  this  set  of  trajectories  is  0.4542.  The  time  required  to  solve 
ProbB  was  28.71  hours,  including  0.33  hours  to  generate  the  target  trajectories. 


Figure  8.  Trajectories  based  on  Algorithm  V.2  on  ProbB.  HVU  trajectory  is  blue. 
Helicopter  trajectory  is  green.  DDG  trajectories  are  black.  Target  trajectories  are 
magenta  and  red. 

In  ProbC  the  HVU  is  under  threat  of  attack  from  L  —  1  target.  The  target  is 
assumed  to  leave  from  one  of  the  two  vertical  sides  of  the  area  of  interest.  The  target 
trajectories  are  conditioned  in  the  same  way  as  they  were  for  ProbA  and  ProbB.  We 
assume  that  the  distributions  for  starting  time  and  starting  location  are  independent 
and  uniform.  The  range  of  the  uniform  distribution  for  starting  location  is  [0,140], 
as  we  again  combine  both  sides  of  the  AOI  into  a  single  segment.  The  range  of  the 
uniform  distribution  for  starting  time  is  [0, 1]. 

The  target  trajectories  are  generated  in  the  same  manner  as  they  were  for 
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ProbA  with  the  parameter  values  given  in  Table  10.  We  note  that  we  now  consider  a 
high-speed  target,  with  a  maximum  speed  of  60  kts. 


Table  10.  Target  and  HVU  parameter  values. 


^0 

v f 

^ min 

v 1 

max 

yljtdT 

rr0(0) 

T.o 

x3 

v° 

10  kts 

12  kts 

1  kt 

60  kts 

250  (35,0)  (nm,nm) 

vr/2 

25  kts 

The  HVU  again  has  an  escort  consisting  of  K  =  3  searchers.  The  goal  of  the 
searcher  is  to  minimize  the  probability  of  failing  to  detect  any  of  the  targets  during 
the  planning  horizon  [0,1].  The  searcher  and  detection  rate  parameters  are  the  same 
as  those  given  in  Tables  7  and  8,  respectively. 

The  dimension  of  the  decision  vector,  fj(-),  in  ProbC  is  KN0  +  K  =  603.  The 
resulting  trajectories  from  Algorithm  V.2  are  shown  in  Figure  9.  The  color  codes  for 
the  trajectories  are  the  same  as  described  above  for  Figure  6,  except  that  the  target 
trajectories  are  shown  in  red.  Again,  we  note  that  only  the  set  of  target  trajectories 
based  on  a  starting  time  of  t  —  0  and  all  starting  locations  is  plotted  in  Figure  9. 
The  objective  value,  which  represents  the  probability  that  all  of  the  searchers  fail  to 
detect  any  of  the  targets  during  [0, 1],  for  this  set  of  trajectories  is  0.7979.  The  time 
required  to  solve  ProbC  was  2.59  hours,  including  0.15  hours  to  generate  the  target 
trajectories. 

The  results  in  this  section  show  that  Algorithm  V.2  is  tractable  for  as  many 
as  three  searchers  and  ten  targets.  The  results  also  indicate  that  it  is  beneficial 
for  the  searchers  to  leave  the  vicinity  of  the  HVU  and  seek  out  potential  threats  in 
order  to  improve  the  overall  probability  of  detection  or  expected  number  of  targets 
detected.  The  results  of  ProbC  illustrate  the  effect  of  a  high-speed  target.  The 
helicopter  searcher  does  not  spend  too  much  time  on  one  side  of  the  AOI,  but  rather 
attempts  to  cover  what  it  can  on  one  side  and  then  flies  over  to  the  other  side  of 
the  AOI.  Finally,  we  see  that  obtaining  numerical  solutions  for  ProbA,  ProbB,  and 
ProbC  using  a  fixed  discretization  scheme  requires  a  significant  amount  of  computing 
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Figure  9.  Trajectories  based  on  Algorithm  V.2  on  ProbC.  HVU  trajectory  is  blue. 
Helicopter  trajectory  is  green.  DDG  trajectories  are  black.  Target  trajectories  are 
red. 

time.  There  is  evidence  that  using  an  adaptive  discretization  scheme  can  produce 
solutions  that  are  equivalent  to  those  obtained  using  fixed  discretization  schemes, 
but  at  a  lower  computational  cost;  see,  for  example,  He  &  Polak  (1990)  and  Section 
3.3.3  in  Polak  (1997).  We  consider  an  algorithm  based  on  an  adaptive  discretization 
scheme  in  the  next  section. 

2.  Adaptive  Discretization  Schemes 

In  this  section  we  obtain  numerical  solutions  for  ProbA  using  an  adaptive 
discretization  scheme  based  on  Algorithm  V.4  and  compare  them  to  results  obtained 
for  ProbA  using  the  fixed  discretization  scheme  of  Algorithm  V.2.  The  parameters 
f3i,  fa,  and  fa  used  to  define  what  it  means  for  the  optimality  function  to  be  “close 
enough”  to  zero,  the  successor  function,  and  the  function  M(N)  used  to  specify 
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the  level  of  spatial  discretization  as  a  function  of  the  time  discretization  parameter 
considered  in  this  section  are  summarized  in  Table  11.  The  column  heading  Set  in 
Table  11  refers  to  the  set  of  algorithm  parameters  used  to  obtain  the  solution  for 
ProbA  using  either  Algorithm  V.2  or  V.4,  as  indicated  in  the  table.  We  note  that 
the  expressions  for  Mt(N)  given  for  Algorithm  V.4  indicate  that  the  initial  spatial 
discretization  parameters  are  (5,  5)  and  that  they  are  each  increased  by  adding  2  (for 
Sets  4  and  6)  or  4  (for  Sets  5  and  7)  to  the  previous  value  every  time  the  value  of  the 
time  discretization  parameter  is  doubled  according  to  k(N). 


Table  11.  Algorithm  parameters  used  to  obtain  solutions  for  ProbA,  where  K  =  3, 
L  =  10,  and  i  =  1,2.  For  all  Sets,  fj0  =  (7t/4,  7t/2,  7t/2,  0). 


Set 

Algo. 

N0 

M0 

fa 

fa 

fa 

k{N) 

Mi{N) 

1 

V.2 

80 

(13,13) 

N/A 

N/A 

N/A 

N/A 

N/A 

2 

V.2 

120 

(17,17) 

N/A 

N/A 

N/A 

N/A 

N/A 

3 

V.2 

320 

(25,25) 

N/A 

N/A 

N/A 

N/A 

N/A 

4 

V.4 

10 

N/A 

10"9 

5  x  10~10 

5  x  10"10 

N'  =  2  N 

5  +  21og2()))) 

5 

V.4 

10 

N/A 

10~9 

5  x  10~10 

5  x  10"10 

N'  =  2  N 

5  +  41og2(^) 

6 

V.4 

10 

N/A 

10"8 

5  x  10”9 

5  x  10"9 

N'  =  2  N 

5  +  21og2(^) 

7 

V.4 

10 

N/A 

10~8 

5  x  10"9 

5  x  10~9 

N'  =  2  N 

5  +  41og2(^) 

The  solutions  we  obtain  are  based  on  the  same  scenario  described  in  Section 
V.B.l  for  ProbA.  The  target  trajectories  are  conditioned  and  generated  as  described 
above  for  ProbA  using  parameter  values  given  in  Table  6.  The  goal  of  the  K  =  3 
searchers  is  to  minimize  the  probability  that  they  all  fail  to  detect  any  of  the  targets 
during  [0, 1].  The  searcher  and  detection  rate  parameters  are  the  same  as  those  given 
in  Tables  7  and  8,  respectively. 

The  data  collected  by  using  Algorithms  V.2  and  V.4  is  shown  in  Figure  10. 
The  objective  function  values  for  all  seven  algorithm  parameter  sets  were  evaluated 
using  a  “high  precision1'  discretization  level  of  N  =  2560  and  M  =  (91,91)  to  ensure 
they  were  all  compared  against  a  common  standard,  ft  should  be  noted  that  when 
using  the  adaptive  discretization  scheme  it  is  possible  for  the  objective  function  value 
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to  go  up  when  the  algorithm  increases  the  values  of  the  discretization  parameters.  An 
example  of  this  behavior  is  seen  for  Set  6  after  approximately  10000  seconds,  where  the 
algorithm  increases  N  from  40  to  80  and  M  from  (9,9)  to  (11, 11).  Based  on  Figure 
10,  it  is  clear  that  all  of  the  adaptive  precision  schemes  eventually  achieve  a  solution 
quality  within  1.42%  of  the  best  solution  obtained  using  one  of  the  fixed  discretization 
schemes.  While  the  fixed  discretization  scheme  for  Set  2  does  better  than  the  adaptive 
discretization  schemes  after  approximately  4000  seconds,  it  is  important  to  note  that 
all  of  the  adaptive  discretization  schemes  reach  an  objective  value  between  0.385  and 
0.39  more  than  a  thousand  seconds  before  the  fixed  discretization  scheme  for  Set  2 
does.  It  is  also  important  to  note  that  all  of  the  adaptive  schemes  perform  better  than 
the  fixed  schemes  based  on  Sets  1  and  3.  This  implies  that  it  is  possible  to  do  better 
for  a  specific  problem  if  you  happen  to  select  a  “good”  fixed  discretization  scheme  at 
the  outset,  but  doing  so  without  any  prior  knowledge  about  the  problem  is  difficult. 
The  adaptive  schemes  are  fairly  robust  in  the  sense  that  they  do  well  regardless  of  the 
initial  choice  of  parameter  values.  The  results  in  Figure  10  indicate  that  the  adaptive 
precision  schemes  for  Sets  5  and  7  offered  the  best  overall  performance  compared  to 
the  other  adaptive  discretization  schemes. 

Figure  11  shows  the  resulting  trajectories  from  Algorithms  V.2  and  V.4.  The 
color  codes  for  the  trajectories  are  the  same  as  described  above  for  Figure  6.  Again, 
we  note  that  only  the  set  of  target  trajectories  based  on  a  starting  time  of  t  —  0 
and  all  starting  locations  is  plotted  in  Figure  11.  It  is  clear  from  the  plots  in  Figure 
11  that  the  overall  “shape”  of  the  final  searcher  trajectories  is  the  same  regardless 
of  the  algorithm  and  parameters  used  to  obtain  the  solution,  with  the  caveat  that 
the  helicopter  searcher  trajectory  is  the  mirror  image  when  it  goes  to  the  opposite 
side  of  the  AOI.  The  times  used  to  plot  the  trajectories  in  Figure  11  for  the  fixed 
discretization  schemes  (Sets  1-3)  were  selected  so  that  enough  computational  time 
had  elapsed  to  allow  the  solution  to  stabilize.  For  the  adaptive  discretization  schemes 
(Sets  4-7),  the  trajectories  were  only  captured  when  the  algorithm  increased  the 


138 


Figure  10.  Comparison  of  fixed  and  adaptive  precision  schemes  using  Algorithms  V.2 
and  V.4,  respectively.  For  Sets  1  and  2  computation  was  terminated  between  10000 
and  15000  seconds  because  the  solution  had  stabilized.  Because  Set  3  did  not  begin 
to  stabilize  until  after  20000  seconds,  the  horizontal  axis  was  extended. 


discretization  level.  Because  of  this  limitation,  the  times  used  to  plot  the  trajectories 
in  Figure  11  for  the  adaptive  discretization  schemes  were  selected  based  on  the  largest 
computational  time  data  that  was  available. 
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Figure  11.  Trajectories  for  ProbA  based  on  Algorithms  V.2  and  V.4.  Top  Row  Left: 
Set  1  after  10339  seconds,  N  =  80,  M  =  (13,13).  Top  Row  Right:  Set  2  after 
12201  seconds,  N  =  120,  M  =  (17, 17).  Second  Row  Left:  Set  3  after  25147  seconds, 
N  =  320,  M  =  (25,25).  Second  Row  Right:  Set  4  after  29852  seconds,  N  =  80, 
M  =  (11,11).  Third  Row  Left:  Set  5  after  23152  seconds,  N  =  40,  M  =  (13,13). 
Third  Row  Right:  Set  6  after  22167  seconds,  N  =  80,  M  =  (11,11).  Bottom  Row 
Middle:  Set  7  after  39998  seconds,  N  =  80,  M  =  (17, 17). 
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3.  Real-Time  Methods 

Although  the  adaptive  discretization  schemes  discussed  in  Section  V.B.2  offer 
improved  run-time  performance  over  fixed  discretization  schemes,  they  are  still  not 
fast  enough  to  be  implemented  as  real-time  solution  methods  for  use  onboard  a  UAS 
or  USV.  In  this  section  we  consider  three  heuristic  approaches  that  have  the  potential 
to  be  used  as  real-time  solution  methods  onboard  unmanned  vehicles.  This  section 
begins  with  a  discussion  of  how  we  implement  the  three  heuristic  methods.  We  then 
give  a  description  of  how  the  problem  instances  to  be  solved  by  these  three  methods 
were  generated.  Next  we  provide  numerical  results  for  the  three  methods.  Finally,  we 
state  our  conclusions  based  on  the  numerical  results  achieved  by  the  three  methods. 

a.  Heuristic  Methods 

We  consider  three  different  heuristic  methods  in  this  section  that  can 
be  used  to  solve  generalized  optimal  control  problems.  For  simplicity  we  assume  that 
we  have  only  a  single  searcher,  and  we  therefore  omit  the  superscript  notation  for 
the  searcher  number.  It  should  be  noted  that  the  development  that  follows  could  be 
trivially  extended  to  include  multiple  searchers.  The  first  method  we  consider  uses 
Algorithm  V.2.  The  second  and  third  methods  are  based  on  fitting  polynomials  to 
determine  optimal  searcher  trajectories.  The  first  polynomial-based  method  attempts 
to  optimize  directly  over  the  coefficients  of  the  polynomials,  while  the  second  poly¬ 
nomial  based  method  uses  an  indirect  method  similar  to  that  found  in  Yakimenko 
(2000)  and  Ghabcheloo  et  al.  (2009).  The  first  polynomial-based  method  solves  the 
following  modified  version  of  ( GTPC ). 
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where  WtJ  are  the  weights  for  Simpson’s  rule,  x  (n)  =  ?  Oii  j  are 

the  discretization  points  at  which  the  integrand  is  evaluated,  d  is  the  order  of  the 
polynomials,  v  is  the  searcher’s  speed,  and  xi(0)  and  £2(0)  are  the  initial  position  of 
the  searcher.  We  note  that  (V.24)  constrains  the  searcher’s  speed  to  be  within  5%  of 
v. 

We  then  use  the  following  algorithm  to  solve  (GTPc,dir). 

Algorithm  V.5.  Approximately  solves  (GT Pc,dir) . 

Data:  N0  G  A /”,  M0  G  kf3  x  N3,  a0  G  M2  x  M2  x  ...  x  M2. 

Parameter:  d  G  N,  d  >  1. 

Step  0.  Set  N  =  N0,  M  =  M0,  and  a  =  oq. 

Step  1.  Generate  {a*}?20  using  a.i+ 1  G  A((GT  Pc,dir) ,  a*).  □ 

The  second  polynomial  based  method  uses  an  indirect  method  to  deter¬ 
mine  the  coefficients  of  the  polynomials.  We  use  the  term  indirect  because  instead  of 
directly  optimizing  over  the  polynomial  coefficients,  we  instead  optimize  over  termi¬ 
nal  conditions  which  can  be  used  to  obtain  the  polynomial  coefficients.  The  method 
used  is  similar  to  that  described  in  Yakimenko  (2000)  and  Ghabcheloo  et  al.  (2009), 
except  that  we  omit  deconffiction  of  trajectories.  In  our  problem  we  also  assume  the 
searcher  travels  at  constant  velocity,  so  the  searcher’s  acceleration  is  zero  over  the 
entire  planning  horizon.  The  description  of  the  method  that  follows  is  similar  to  that 
found  in  Yakimenko  (2000)  and  Ghabcheloo  et  al.  (2009),  but  is  included  here  for 
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the  sake  of  completeness.  We  denote  by  x(t)  =  (x\  (r) ,  x2  (r))T  the  desired  path 
followed  by  the  searcher,  parameterized  by  the  virtual  arc  r  e  [0 ,77],  where  77  is  the 
total  virtual  arc  length  between  the  initial  and  final  positions  of  the  searcher.  We 
represent  the  state  as  a  function  of  time  using  the  same  notation  as  the  state  as  a 
function  of  the  virtual  arc  length.  The  meaning  should  be  clear  from  the  context.  We 
represent  the  coordinates  x\  and  X2  by  algebraic  polynomials  of  degree  d  given  by 

d 

xj(r)  =  Y,^r\  j  —  1,  2,  (V.28) 

i= 0 

where  r'  indicates  that  r  is  raised  to  the  ith  power.  The  degree  d  of  the  polynomials 
Xj(t),  j  =  1,  2  is  related  to  the  number  of  boundary  conditions  that  must  be  satisfied. 
In  the  formulation  that  follows,  we  use  the  prime  sign  /  to  indicate  d/dr,  //  to  indicate 
the  second  derivative  operator,  and  ///  to  indicate  the  third  derivative  operator.  As 
before,  we  use  dot  symbols  above  variables  to  indicate  derivatives  with  respect  to 
time,  t.  Given  77  and  terminal  constraints  Xj(0),  x'-(0),  x"(0),  x'"(0),  Xj(rf ),  £'(77), 
x"(r/),  and  Xj'^Tf),  for  j  =  1,2,  the  coefficients  of  the  seventh-order  polynomial  can 
be  computed  from 
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(V.29) 

In  order  to  express  the  relationship  between  r  and  t,  we  define  A  (r)  =  Then  the 
temporal  and  spatial  derivatives  of  x  satisfy 

x  =  Xx1.  (V.30) 
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As  in  Yakimenko  (2000)  and  Ghabcheloo  et  al.  (2009)  we  choose  A  to  be  an  affine 
function  of  r,  that  is,  A(r)  =  Ao  +  AjrjA"r,  with  Ao  =  ||i(0)||,  and  A/  =  ||i(l)||. 
By  integrating  t  =  A  (r),  the  virtual  arc  r  and  time  t  are  related  by  the  following 
equations 


Tf 


A/-a() 
log(A//A0)  ’ 


r 

Tf 


t, 


Aq 


A/-  Ao 


if  Xf  =  A0 
if  A/  7^  A0 

if  A/  =  A0 
if  Xf  ^  A0 


(V.31) 


(V.32) 


The  second  polynomial  based  method  solves  the  following  modified  version  of  ( GTPC ), 
where  Table  12  gives  a  summary  of  the  relationship  between  the  boundary  conditions 
needed  to  solve  (V.29)  and  the  decision  vector  b. 


Table  12.  Relationship  between  decision  vector  and  boundary  conditions  for  indirect 
polynomial  method. 
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b2 
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b  {  i- 1  3= 1 
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N- 1  L 

EE1' 

7=0  1=1 


d 

s.t.  x-j (t )  =  ^  a}r\  j  =  1,  2,  Vr  e  [0,  rf] 

i= 0 

n  —  0.05n  <  A(r)  ||o/(t)||  <  v  +  0.05n,  Vr  e  [0,  tj\ 

In  problem  (GT pc>m(for) ;  t'G.j  are  the  weights  for  Simpson’s  rule,  are  the  discretiza¬ 
tion  points  at  which  the  integrand  is  evaluated,  d  is  the  order  of  the  polynomials,  and 
v  is  the  searcher’s  speed.  The  total  arc  length  r/  is  computed  using  (V.31)  and  the 
spatial  paths  x(r)  and  speed  profiles  x  are  given  by  (V.29)  and  (V.30),  respectively. 
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In  order  to  discretize  r,  we  set  t  —  7  =  0, —  1  in  (V.32).  We  note  that 

for  j  =  1,2,  £j(0)  are  the  starting  coordinates  for  the  searcher,  x'j(0)  =  x'J(rf)  =  0, 
and  x'  (0)  are  determined  by  solving  the  initialization  problem  described  in  Algorithm 
V.6. 

We  then  use  the  following  algorithm  to  solve  (GT pc>mcfer). 

Algorithm  V.6.  Approximately  solves  (GT  Pc'indlT) . 

Data:  N0  G  A f ,  M0  £  N3  x  N3,  rjo  G  HC;jv0- 

Parameter:  d  G  N,  d  >  1. 

Step  0.  Set  N  =  N0,  M  =  Ado,  and  fjo  =  Wn(t]o). 

Step  1.  Generate  {?7;}”=0  using  rji+i  G  A((GTPcNM),  rii),  where  n  is  the  first  iterate 
satisfying  a  given  optimality  tolerance. 

Step  2.  Compute  xl  from  fjn. 

Step  3.  Set  bj  =  Xj(  1),  j  =  1,2,  &3  =  cos(a:3(l)),  64  =  sin(a;3(l)),  and  bj  =  0, 
J  —  5,  6,  7,  8. 

Step  4.  Generate  {h}^  using  bi+ 1  G  A((GT Pc'mdir) ,  bi).  □ 

In  Step  1  of  Algorithm  V.6  we  are  using  Algorithm  V.2  to  solve 
(GTPCNM),  and  stopping  it  after  satisfying  a  given  optimality  tolerance.  In  Step 
2  of  Algorithm  V.6  we  use  the  solution,  fjn,  from  Step  1  to  compute  the  searcher’s 
trajectory.  In  Step  3  of  Algorithm  V.6  we  set  bj  =  Xj(  1),  j  =  1,  2,  where  (xi(l),  £2(1)) 
is  the  final  position  of  the  searcher  we  calculated  in  Step  2.  We  also  use  fjn  from  Step  2 
to  determine  the  searcher’s  final  heading,  which  we  resolve  into  components  to  obtain 
values  for  63  and  64.  We  initialize  the  remaining  components  of  b  to  zero  and  move 
on  to  Step  4,  where  we  determine  an  optimized  value  for  b. 

b.  Problem  Instances 

All  of  the  problem  instances  are  based  on  a  HVU  operating  in  a  70  nm 
by  70  nm  area  of  interest  using  a  receding  horizon  approach,  and  a  one  hour  planning 
horizon.  For  these  instances,  the  HVU  is  under  threat  of  attack  from  L  =  10  targets. 
The  target  trajectories  are  conditioned  and  generated  as  described  above  for  ProbA. 
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The  parameter  values  used  to  generate  the  target  trajectories  are  given  in  Table  6. 
We  again  assume  that  the  separation  in  starting  position  between  members  of  the 
swarm  is  1.5  nm.  We  assume  that  the  HVU  has  a  single  helicopter  escort,  whose 
goal  is  to  minimize  the  probability  that  it  fails  to  detect  any  of  the  targets  during 
the  planning  horizon  [0, 1].  With  the  exception  of  £i(0)  and  ^(O),  which  are  given 
in  Table  13,  the  searcher  and  detection  rate  parameters  are  the  same  as  those  given 
in  Tables  7  and  8,  respectively,  with  K  —  1. 

We  randomly  generate  the  starting  position  for  the  searcher,  given  in 
columns  two  and  three  of  Table  13.  The  xi(0)  value  was  selected  from  the  range 
[20,50],  and  the  2:2(0)  value  was  selected  from  the  range  [0,30].  We  let  the  vertical 
starting  position  for  the  targets  on  the  side  (or  sides)  of  the  area  of  interest  be  a 
continuum  between  [0, 140]  and  again  randomly  select  the  lower  and  upper  bounds 
for  the  starting  position  of  the  targets.  The  generated  values  are  given  in  columns 
four  and  five  of  Table  13. 

Table  13.  Real-time  problem  instances.  For  all  instances,  problem  class  is  ( GTPC ), 
K  =  1,  and  L  =  10. 


Instance 

2h(0) 

x2(0) 

Target  LB 

Target  LIB 

Side  Scenario 

ProbD 

34.56 

24.01 

19.86 

59.05 

LHS 

ProbE 

47.47 

11.77 

23.97 

91.77 

Both 

ProbF 

23.81 

27.4 

88.53 

134.05 

RHS 

c.  Numerical  Results 

In  this  section,  we  provide  numerical  results  obtained  using  Algorithms 
V.2,  V.5,  and  V.6  to  solve  the  problem  instances  given  in  Table  13.  We  implement  and 
run  all  Algorithms  in  MATLAB  7.11.0  on  the  same  PC  described  in  Section  V.B.l, 
again  using  SNOPT  with  default  major  and  minor  optimality  tolerance  as  the  stop¬ 
ping  criteria  except  as  noted  below.  We  separate  the  results  for  each  problem  instance 
into  two  tables,  one  based  on  the  fixed  discretization  method  given  in  Algorithm  V.2, 
and  one  based  on  the  polynomial  methods  given  in  Algorithms  V.5  and  V.6.  We  also 
provide  a  figure  for  each  problem  instance  that  shows  the  resulting  trajectories  from 


146 


the  three  solution  methods.  The  result  from  Algorithm  V.2  in  Figures  12,  13,  and 
14  uses  the  first  set  of  parameters  that  achieve  an  objective  function  value  smaller 
than  either  of  the  objective  function  values  achieved  by  Algorithms  V.5  and  V.6.  We 
plot  all  results  using  a  value  of  IV  =  320  and  M  =  (25,  25)  to  ensure  a  common 
standard.  This  need  to  “level  the  playing  held”  creates  two  different  issues  based 
on  the  solution  methodologies.  The  results  given  in  Figures  12,  13,  and  14  are  from 
solutions  generated  with  N  values  smaller  than  320.  As  a  result,  the  control  inputs 
for  the  searcher  resulting  from  Algorithm  V.2  are  linearly  interpolated  to  N  =  320  to 
generate  the  plots.  Algorithms  V.5  and  V.6  generate  polynomial  coefficients,  so  there 
is  no  issue  with  the  N  =  320  used  for  plotting  being  larger  than  the  N  value  used  to 
generate  the  coefficients.  There  is  an  issue,  however,  when  it  comes  to  ensuring  that 
the  searcher  maintains  a  constant  speed  equal  to  v  along  its  trajectory.  We  generate 
the  plots  for  Algorithms  V.5  and  V.6  using  the  path  for  the  searcher  defined  by  the 
polynomial  coefficients,  but  we  ensure  that  the  distance  traveled  between  each  of  the 
N  =  320  searcher  waypoints  is  limited  to  where  v  is  the  velocity  of  the  searcher. 

Tables  14,  16,  and  18  provide  results  from  Algorithm  V.2.  The  initial 
control  input  for  the  searcher  is  the  zero  vector  of  appropriate  length,  and  the  initial 
heading  for  the  searcher  is  chosen  based  on  the  side  scenario.  For  the  left  hand  side 
and  both  side  scenarios,  the  initial  heading  for  the  searcher  is  37t/4.  For  the  right 
hand  side  scenario,  the  initial  heading  for  the  searcher  is  7t/4.  Tables  14,  16,  and  18 
list  the  values  of  N  and  M  used  to  run  Algorithm  V.2,  but  all  of  the  objective  function 
values  in  row  three  of  these  tables  are  evaluated  using  a  common  discretization  level  of 
N  =  320  and  M  =  (25,  25).  Row  four  in  Tables  14,  16,  and  18  gives  the  time  required 
for  the  optimization  only,  and  does  not  include  time  to  build  the  target  trajectories. 

To  implement  Algorithm  V.5,  we  use  d  =  7.  The  initial  input  for  the 
polynomial  coefficients  is  chosen  based  on  the  side  scenario.  For  the  left  hand  side 
scenario,  all  of  the  coefficients  are  zero  with  the  exception  of  a\  which  is  set  to  a  value 
of  — Xi(0).  This  results  in  an  initial  horizontal  line  trajectory  for  the  searcher  from  its 
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starting  location  of  (xi(0),  £2(0))  and  ending  at  (0,  24.01).  For  the  both  side  scenario, 
we  set  a\  =  —88.98,  of  =  —0.37,  af  =  0.03,  al  =  45.64,  a\  =  —0.40,  a\  =  0.04, 
and  all  the  other  coefficients  equal  to  zero.  This  results  in  an  initial  straight  line 
trajectory  for  the  searcher  with  slope  -0.507,  beginning  at  (oq(0), rr2(0))  and  ending 
at  (—41.85,57.05).  For  the  right  hand  side  scenario,  all  of  the  coefficients  are  zero 
with  the  exception  of  a\  which  is  set  to  a  value  of  70  —  aq(0)  =  46.19.  This  results 
in  an  initial  horizontal  line  trajectory  for  the  searcher  from  its  starting  location  of 
(aq(0), x2(0))  and  ending  at  (70,27.4).  Tables  15,  17,  and  19  list  the  values  of  N 
and  M  used  to  run  Algorithm  V.5,  but  all  of  the  objective  function  values  in  row 
four  of  these  tables  are  evaluated  using  a  common  discretization  level  of  N  =  320 
and  M  =  (25,25).  Row  five  in  Tables  15,  17,  and  19  gives  the  time  required  for  the 
optimization  only,  and  does  not  include  time  to  build  the  target  trajectories.  We  note 
that  for  ProbD  the  stopping  criteria  for  Algorithm  V.5  is  not  the  default  major  and 
minor  optimality  criteria  for  SNOPT.  Because  we  are  interested  in  real-time  methods, 
we  also  set  a  maximum  number  of  major  iterations.  In  the  case  of  ProbD,  SNOPT 
reaches  the  700  major  iteration  limit  prior  to  satisfying  its  default  optimality  criteria. 
This  is  the  reason  why  the  searcher  trajectory  shown  in  middle  plot  of  Figure  12  goes 
so  far  to  the  left,  beyond  the  target  trajectories.  The  algorithm  was  stopped  before 
it  could  finish  optimizing  the  trajectory  to  include  as  much  probability  “mass”  as 
possible,  while  still  satisfying  the  velocity  constraints. 

To  implement  Algorithm  V.6,  we  use  cl  =  7.  Because  Algorithm  V.6  is 
effectively  initialized  by  Algorithm  V.2,  we  again  use  the  zero  vector  of  appropriate 
length  as  the  initial  control  input  for  the  searcher.  For  the  left  hand  side  and  both 
side  scenarios,  the  initial  heading  for  the  searcher  is  37t/4.  For  the  right  hand  side 
scenario,  the  initial  heading  for  the  searcher  is  7t/4.  Tables  15,  17,  and  19  list  the 
values  of  N  and  M  used  to  run  Algorithm  V.6,  but  all  of  the  objective  function 
values  in  row  four  of  these  tables  are  evaluated  using  a  common  discretization  level  of 
N  =  320  and  M  =  (25,  25).  Row  five  in  Tables  15,  17,  and  19  gives  the  time  required 
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for  the  initialization  and  optimization  only,  and  does  not  include  time  to  build  the 
target  trajectories.  We  note  that  the  stopping  criteria  for  Algorithm  V.6  is  not  the 
default  major  and  minor  optimality  criteria  for  SNOPT.  Again,  based  on  our  desire 
to  find  real-time  methods  we  set  the  stopping  criteria  as  15  major  iterations.  This  is 
the  reason  why  the  searcher  trajectories  shown  in  the  right  plots  of  Figures  12  and  14 
go  so  far  beyond  the  target  trajectories.  The  algorithm  was  stopped  before  it  could 
finish  optimizing  the  trajectories  to  include  as  much  probability  “mass”  as  possible, 
while  still  satisfying  the  velocity  constraints.  We  note  that  clue  to  the  complicated 
relationship  between  the  objective  function  and  the  decision  vector,  we  use  finite 
differences  in  SNOPT  to  estimate  the  gradients.  Finally,  we  note  that  we  initialize 
ProbE  using  IV  =  20  and  M  =  (9,9)  and  not  N  =  10  and  M  =  (5,5).  This  is 
because  this  method  is  highly  dependent  on  a  good  initial  input  for  the  searcher’s 
final  position,  and  the  lower  levels  of  discretization  do  not  provide  a  solution  where 
the  searcher  goes  to  both  sides  of  the  AOI  to  look  for  the  targets. 


149 


Table  14.  Algorithm  V.2  results  for  ProbD. 


N 

10 

15 

20 

25 

30 

35 

40 

320 

M 

(5,5) 

(7J) 

(9,9) 

(ii,ii) 

(13,13) 

(15,15) 

(15,15) 

(25,25) 

Obj. 

0.0476 

0.0294 

0.0324 

0.0206 

0.0097 

0.0118 

0.0092 

0.0088 

Time 

(sec.) 

10.67 

48.76 

141.77 

203.48 

361.53 

612.03 

884.68 

16223.48 

Table  15.  Algorithm  V.5  and  V.6  results  for  ProbD. 


Direct 

Indirect 

N 

10 

10 

M 

(5,5) 

(5,5) 

Obj. 

0.0246 

0.0927 

Time  (sec.) 

317.40 

176.50 

Figure  12.  Trajectories  for  ProbD  based  on  Algorithms  V.2,  V.5,  and  V.6.  Left: 
After  203  seconds,  Algorithm  V.2,  N  =  25,  M  =  (11, 11)  for  solution,  N  =  320,  M  = 
(25,25)  for  plot.  Middle:  After  317  seconds,  Algorithm  V.5,  N  =  320,  M  =  (25,25). 
Right:  After  176  seconds,  Algorithm  V.6,  N  =  320,  M  =  (25,25). 
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Table  16.  Algorithm  V.2  results  for  ProbE. 


N 

10 

15 

20 

25 

30 

35 

40 

320 

M 

(5,5) 

(7J) 

(9,9) 

(ii,ii) 

(13,13) 

(15,15) 

(15,15) 

(25,25) 

Obj. 

0.5161 

0.5022 

0.2767 

0.2715 

0.2715 

0.2622 

0.2593 

0.2555 

Time 

(sec.) 

8.29 

32.60 

52.51 

93.55 

175.11 

284.54 

431.95 

43000.29 

Table  17.  Algorithm  V.5  and  V.6  results  for  ProbE. 


Direct 

Indirect 

N 

10 

20 

M 

(5,5) 

(9,9) 

Obj. 

0.3526 

0.2918 

Time  (sec.) 

66.05 

750.00 

Figure  13.  Trajectories  for  ProbE  based  on  Algorithms  V.2,  V.5,  and  V.6.  Left:  After 
53  seconds,  Algorithm  V.2,  N  =  20,  M  =  (9,9)  for  solution,  N  =  320,  M  =  (25,  25) 
for  plot.  Middle:  After  66  seconds,  Algorithm  V.5,  N  =  320,  M  =  (25,25).  Right: 
After  750  seconds,  Algorithm  V.6,  N  =  320,  M  =  (25,25). 
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Table  18.  Algorithm  V.2  results  for  ProbF. 


N 

10 

15 

20 

25 

30 

35 

40 

320 

M 

(5,5) 

(7J) 

(9,9) 

(ii.ii) 

(13,13) 

(15,15) 

(15,15) 

(25,25) 

Obj. 

0.0553 

0.0330 

0.0246 

0.0265 

0.0249 

0.0237 

0.0234 

0.0216 

Time 

(sec.) 

9.29 

39.40 

74.77 

225.61 

464.91 

818.13 

1008.03 

35608.20 

Table  19.  Algorithm  V.5  and  V.6  results  for  ProbF. 


Direct 

Indirect 

N 

10 

10 

M 

(5,5) 

(5,5) 

Obj. 

0.0734 

0.0421 

Time  (sec.) 

67.85 

185.19 

Figure  14.  Trajectories  for  ProbF  based  on  Algorithms  V.2,  V.5,  and  V.6.  Left:  After 
39  seconds,  Algorithm  V.2,  N  —  15,  M  —  (7,  7)  for  solution,  N  =  320,  M  =  (25,  25) 
for  plot.  Middle:  After  68  seconds,  Algorithm  V.5,  N  =  320,  M  =  (25,25).  Right: 
After  185  seconds,  Algorithm  V.6,  N  =  320,  M  =  (25,25). 
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d.  Conclusions 

Based  on  the  results  presented  in  this  section,  it  appears  that  the 
method  which  is  best  suited  to  provide  real-time  solutions  is  Algorithm  V.2.  The 
fixed-precision  method  implemented  in  Algorithm  V.2  consistently  provides  solutions 
with  the  best  objective  values  and  the  lowest  computational  times.  Table  20  summa¬ 
rizes  the  comparison  between  the  different  methods.  To  compute  the  values  in  Table 
20,  we  assume  that  the  objective  value  obtained  from  Algorithm  V.2  with  N  =  320 
and  M  =  (25,  25)  is  the  “correct”  value.  For  ProbD,  we  use  the  objective  values 
from  Algorithm  V.2  for  N  —  25  and  M  =  (11, 11)  and  N  —  20  and  M  =  (9,9)  to 
compare  with  Algorithms  V.5  and  V.6,  respectively.  For  ProbE,  we  use  the  objective 
value  from  Algorithm  V.2  for  N  =  20  and  M  =  (9,  9)  to  compare  with  Algorithms 
V.5  and  V.6.  For  ProbF,  we  use  the  objective  value  from  Algorithm  V.2  for  N  =  15 
and  M  =  (7,  7)  to  compare  with  Algorithms  V.5  and  V.6.  The  At  columns  in  Table 
20  give  the  additional  time  in  seconds  necessary  for  the  Algorithms  V.5  and  V.6  to 
compute  their  respective  solutions.  In  fairness,  it  should  again  be  stressed  that  the 
implementation  of  Algorithm  V.6  utilized  finite  differences  to  estimate  the  required 
gradients.  This  puts  Algorithm  V.6  at  a  disadvantage  over  the  others,  although  we 
attempt  to  mitigate  this  effect  by  stopping  the  algorithm  after  only  15  major  itera¬ 
tions.  Future  studies  should  be  done  with  the  gradients  computed  explicitly  to  ensure 
a  more  balanced  comparison. 


Table  20.  Comparison  of  real-time  methods. 


Algo. 

ProbD 
%  Error 

At 

ProbE 
%  Error 

At 

ProbF 
%  Error 

At 

V.2 

134.8%/269.3% 

N/A 

8.29% 

N/A 

53.10% 

N/A 

V.5 

180.90% 

113.92 

37.98% 

13.55 

239.99% 

28.46 

V.6 

957.20% 

34.73 

14.19% 

697.49 

94.92% 

145.8 
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VI. 


CONCLUSIONS  AND  FUTURE  WORK 


A.  CONCLUSIONS 

Continuous  timc-and-space  search  problems  arise  in  numerous  applications 
such  as  anti-submarine  warfare,  search  and  rescue  operations,  and  protection  of 
HVU’s  from  small  boat  attack.  We  consider  the  problem  of  detecting  targets  that 
seek  to  harm  a  HVU,  and  formulate  it  as  a  generalized  optimal  control  problem. 
While  the  theory  that  we  develop  allows  for  more  general  cases,  we  assume  that 
the  targets  follow  deterministic  trajectories,  given  realizations  of  random  variables 
that  provide  information  about  their  initial  conditions.  We  consider  two  classes  of 
problems,  ( GTP )  and  ( ITP ),  based  on  whether  the  targets  act  in  a  coordinated  or 
independent  manner,  respectively.  For  both  classes,  we  consider  objective  functions 
that  minimize  the  probability  that  all  of  the  searchers  fail  to  detect  any  of  the  targets 
during  the  planning  horizon  and  maximize  the  expected  number  of  targets  detected 
during  the  planning  horizon. 

We  develop  discretization  schemes  to  solve  (GTP)  and  (ITP)  problems,  and 
show  that  the  finite  dimensional  problems  are  consistent  approximations  to  their  in¬ 
finite  dimensional  counterparts.  We  then  use  these  discretization  schemes  to  develop 
implementable  algorithms  that  can  be  used  to  solve  the  problems  (GTP),  (GTPC), 
(GTPe),  (GTPc,e),  (ITPP),  (ITPC,P),  ( ITPe ),  and  (ITPc,e).  We  provide  numerical 
examples  to  show  that  our  algorithms  are  tractable  for  as  many  as  three  searchers 
and  ten  targets.  For  (GTPC)  we  compare  an  algorithm  based  on  fixed  discretization 
schemes  to  an  algorithm  based  on  adaptive  discretization  schemes.  Our  results  indi¬ 
cate  that  both  methods  produce  equivalent  solutions,  but  the  adaptive  discretization 
schemes  require  less  computational  time  to  acheive  them. 

We  compare  three  heuristic  approaches  that  have  the  potential  to  be  used 
as  real-time  solution  methods  onboard  unmanned  vehicles.  We  use  two  different 
polynomial  based  approaches,  and  one  fixed-precision  method  to  solve  three  randomly 
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generated  problem  instances.  For  these  problem  instances,  the  fixed-precision  method 
offers  the  best  overall  performance,  ft  consistently  provides  solutions  with  the  best 
objective  values  and  the  lowest  computational  times. 

We  also  develop  rate  of  convergence  results  for  approximations  to  infinite  di¬ 
mensional  problems  that  involve  a  single  discretization  parameter  for  the  space  of 
decision  variables  and  two  discretization  parameters  for  the  objective  function  as  a 
computational  budget  b  tends  to  infinity.  We  note  that  the  generalized  optimal  control 
problems  we  define  in  Chapter  111  are  examples  of  these  infinite  dimensional  problems. 
We  show  that  superlinear  and  linear  algorithms  can  achieve  the  same  theoretical  rate 
of  convergence  as  that  achieved  by  the  “ideal”  case  of  a  finitely  convergent  algorithm. 
We  also  identify  specific  optimal  discretization  policies  for  both  the  superlinear  and 
linear  cases  that  achieve  this  best  possible  rate.  Our  analysis  indicates  that  if  a  linear 
or  superlinear  optimization  algorithm  is  used  to  solve  the  finite  dimensional  optimal 
control  problem,  with  Euler’s  method  used  to  numerically  solve  the  differential  equa¬ 
tions  and  Simpson’s  rule  used  to  numerically  approximate  the  spatial  integration, 
then  the  best  possible  rate  of  convergence  is  b~2^ .  If  a  second-order  Runge-Kutta 
method  is  used  instead  of  Euler’s  method  to  numerically  solve  the  differential  equa¬ 
tions,  then  the  best  possible  rate  of  convergence  is  b~l .  If  a  fourth-order  Runge-Kutta 
method  is  used  instead  of  Euler’s  method  to  numerically  solve  the  differential  equa¬ 
tions,  then  the  best  possible  rate  of  convergence  is  5-4//3.  Finally,  if  an  ideal  method 
is  used  instead  of  Euler’s  method  to  numerically  solve  the  differential  equations,  then 
the  best  possible  rate  of  convergence  is  b~2.  If  “ideal”  methods  are  used  to  solve  the 
differential  equations  as  well  as  evaluate  the  spatial  integration,  there  is  no  benefit 
associated  with  increasing  the  discretization  parameters  N  or  M,  so  b  =  n,  where  n 
is  the  number  of  iterations  of  the  optimization  algorithm.  The  resulting  asymptotic 
rates  for  a  superlinear  optimization  algorithm  with  order  7  G  (1,  00)  and  c  G  (0, 1), 
and  a  linear  optimization  algorithm  with  rate  constant  c  G  (0, 1)  are  c<b  and  cb, 
respectively.  Based  on  our  analysis,  it  appears  that  it  is  possible  to  improve  the  run- 
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time  performance  of  our  algorithms  if  a  higher-order  method  such  as  Runge-Kutta  or 
pseudo-spectra!  is  used  in  place  of  Euler’s  method  to  numerically  solve  the  differential 
equations. 

B.  FUTURE  WORK 

This  dissertation  suggests  a  number  of  possibilities  for  extension  and  future 
work.  The  first  area  we  consider  for  additional  work  is  the  development  of  a  real-time 
solution  method.  As  discussed  in  Section  V.B.3.d,  the  implementation  of  Algorithm 
V.6  used  finite  differences  to  estimate  the  required  gradients.  Future  work  should 
include  explicit  computation  of  all  required  derivatives,  so  that  the  implementation 
of  Algorithm  V.6  is  on  a  more  equal  footing  with  those  of  the  other  two  potential 
real-time  methods.  An  opportunity  for  extension  would  be  to  use  something  other 
than  algebraic  polynomials  as  the  basis  for  the  searcher  trajectories  in  Algorithms 
V.5  and  V.6.  One  suggestion  would  be  to  use  Chebyshev  polynomials.  In  Caporale 
&  Cerrato  (2008)  Chebyshev  polynomials  in  conjunction  with  Chebyshev  nodes  were 
shown  to  provide  excellent  approximations  to  the  solutions  of  linear  partial  differential 
equations,  indicating  that  there  might  be  potential  for  their  use  in  Algorithms  V.5  or 
V.6.  Another  suggestion  would  be  to  use  Bezier  curves  in  Algorithm  V.6.  Because 
Bezier  curves  are  completely  contained  in  the  convex  hull  of  their  control  points,  they 
would  be  particularly  well  suited  to  an  indirect  curve  fitting  method  where  the  control 
points  could  be  determined  based  on  the  low  level  initialization  as  in  Algorithm  V.6. 

As  discussed  in  Section  VI. A  above,  there  is  potential  for  run-time  improve¬ 
ment  if  higher  order  methods  are  used  in  place  of  Euler’s  method  to  numerically  solve 
the  differential  equations.  The  consistent  approximation  theory  and  implementable 
algorithms  developed  in  Chapters  III  and  V,  respectively,  could  be  extended  based 
on  using  Runge-Kutta  or  pseudo-spectral  methods  instead  of  Euler’s  method  to  nu¬ 
merically  solve  the  differential  equations,  however,  we  foresee  numerous  technical 
challenges. 
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In  this  dissertation,  we  consider  two  different  types  of  objective  functions.  An¬ 
other  natural  extension  would  be  to  consider  additional  types  of  objective  functions. 
In  the  sections  that  follow,  we  present  two  alternative  objective  functions  that,  while 
potentially  more  difficult  to  solve,  may  be  of  operational  interest  in  many  situations. 


1.  Minimize  Expected  Time  Until  First  Detection 

We  now  consider  a  new  performance  metric  for  our  optimization  problem  that 
is  based  on  minimizing  the  expected  time  until  the  first  detection  of  all  the  targets. 
In  Chapter  II,  we  derived  (II. 3),  which  is  an  expression  for  the  probability  that  the 
kth  searcher  does  not  detect  the  Ith  target  during  [0,  t],  t  e  [0, 1]  We  define  the  random 
variable  T  as  the  time  of  first  detection  of  the  Ith  target,  and  note  that 

P  ({T  >  t})  =  qk’\t ;  a)  =  exp  (-  jT  rk ’l  (xk (s) ,  yl (s;  a))  ds^j  =  1  -  F(t),  (VI.  1) 


where  F(t)  is  the  cumulative  distribution  function  for  the  random  variable  T.  Then 
for  the  case  of  one  searcher  and  one  target  the  conditional  expected  time  of  first 
detection  given  a  particular  target  trajectory  is  given  by 


E[T\a\  =  / 


(VI.2) 


where  frit)  is  the  probability  density  function  of  the  random  variable  T.  We  note 
that  the  support  of  the  random  variable  T  is  [0, 1],  so  E[T\a\  is  finite.  We  also  note 
that  we  can  use  (VI.  1)  and  integration  by  parts  to  show  that  (VI.2)  can  be  re-written 
in  terms  of  qk,l(t ;  a)  as  follows 

E[T\a]  =  J  qk,l(t-,a)dt  =  j  exp  J  rk’1  (xk(s),yl(s',a)')  ds'j  dt 

=  [  P({T>t})dt=P({T>t})t\l-  ft  ( -hit ))  dt 


tfT\t)dt. 


(VI-3) 


If  we  assume  that  the  searchers  make  independent  detection  attempts,  then  for  the 


158 


case  of  multiple  searchers  the  conditional  expected  time  of  first  detection  given  a 
particular  target  trajectory  is  given  by 


If  we  assume  that  the  random  variables  that  the  target  motion  is  conditioned  upon 
are  independent  across  targets,  then  the  expected  time  of  first  detection  of  the  Ith 
target  is  given  by 


ds  J  dt 

2.  Herding  Formulation 

Throughout  this  dissertation,  we  have  focused  on  detecting  the  potential  threats 
to  the  HVU.  Once  the  potential  threats  are  located,  the  searchers  could  become  de¬ 
fenders  who  now  select  controls  such  that  the  potential  attackers  are  herded  away 
from  the  HVU.  In  a  herding  model,  we  assume  a  different  target  motion  model  where 
the  targets’  motion  is  defined  by  coupled  dynamics,  which  we  describe  in  detail  below. 
Then,  a  natural  metric  for  the  degree  of  success  in  herding  would  be  to  maximize  the 
minimum  distance  between  the  HVU  and  any  attacker  at  any  instance  of  time  in  the 
planning  horizon.  This  herding  success  metric  is  given  by 


ft  (a1)  da1.  (VI. 7) 


(x^t)  -  y[  (t;  a1))2  +  (x°2{t)  -  yl2  (t;  al))Z  ft  (a1)  da1  \  .  (VI.8) 


m2l 


max  <  nun  / 

zfeO  l  M  J al€A  L 

In  order  to  formulate  the  dynamics  between  the  defenders  and  the  attackers, 
we  first  define  some  necessary  quantities.  First,  we  define 


ran9elftd(t)  =  yj (x'ftt)  -  y\  (t;  a1))2  +  {xk2[t)  -  yl2  (t;  a1))2,  (VI.9) 
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which  gives  the  range  between  the  Ith  attacker  and  the  kth  defender  at  time  t,  given 
a  particular  attacker  trajectory.  We  also  define 


rangelHVU(t )  =  \J (x?(t)  -  y[  (t;  a1))2  +  (x%{t)  -  yl2  (t;  a1))2,  (VI.  10) 


which  gives  the  range  between  the  Ith  attacker  and  the  HVU  at  time  t,  given  a 
particular  attacker  trajectory.  Next,  we  define 


9  ad^)  =  tan  1 


Aif)  -  Vl2 
(t)  ~y[  (t;  a1)  J 


(VI-11) 


which  gives  the  angle  between  the  Ith  attacker  and  the  kth  defender  at  time  t,  given 
a  particular  attacker  trajectory.  Finally,  we  define 


—  tan 


s°(*)  ~Vl2 

xi(t)  ~y\  i^a)  J 


(VI. 12) 


which  gives  the  angle  between  the  Ith  attacker  and  the  HVU  at  time  t,  given  a  par¬ 
ticular  attacker  trajectory.  Then,  the  dynamics  of  the  Ith  attacker  given  a  particular 
realization  of  the  random  variable  a  are  given  by 

I< 

v[(t)  = 


COS  dlHVjj{t) 


k=  1 
K 


yl2(t)  =  ~w[Y^ 


range^it ) 

sin^’JW 


+  wt 


[rangelHVU{t)\ 

sin  8lHVu(t) 


k=  i  \rangel£(t) |  [™ngelHVU{t)\ 


2  ’ 


(VI. 13) 


(VI.  14) 


where  w[  and  wl2  are  user  specified  weights  that  define  the  relative  importance  of  the 
two  components  of  the  attacker  dynamics.  The  component  associated  with  w\  is  due 
to  the  herding  effects  of  the  defenders.  In  a  manner  similar  to  the  approach  used  in 
Lu  (2006),  we  assume  that  the  Ith  attacker  moves  in  a  straight  line  away  (hence  the 
minus  sign  in  front  of  w\ )  from  the  weighted  sum  of  “influences”  of  the  k  defenders, 
and  that  the  Ith  attacker’s  velocity  is  bounded  by  the  inverse  of  the  distance  between 
it  and  the  kth  defender  squared.  The  component  associated  with  wl2  is  due  to  the 
attacker’s  desire  to  get  close  to  the  HVU  in  order  to  facilitate  his  attack.  Again,  we 
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assume  that  the  Ith  attacker  moves  in  a  straight  line  towards  (hence  the  plus  sign  in 
front  of  wl2)  the  HVU,  and  that  the  Ith  attacker’s  velocity  is  bounded  by  the  inverse 
of  the  distance  between  it  and  the  HVU  squared. 

One  way  that  w[  and  wl2  could  be  specified  would  be  to  define  a  distance  called 
panic1  that  gives  the  range  from  the  HVU  that  the  Ith  attacker  begins  to  worry  about 
being  detected  by  the  defenders.  In  this  case  w\  and  wl2  would  be  given  by 

,  panic1 

wi  =  - i - 

range  HVU 

„j  _  rangelHVU 

^ 2  •  / 
panic1. 


(VI. 15) 
(VI. 16) 
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VII. 


APPENDIX:  MATHEMATICAL 
BACKGROUND 


Throughout  this  dissertation,  we  consider  the  Lipschitz  continuity  of  functions 
relative  to  the  set  H,  and  the  differentiability  of  functions  on  the  set  H°,  relative  to 
the  set  H  as  defined  in  Polak  (1997).  For  the  sake  of  completeness,  we  include  the 
pertinent  definitions  from  pp.  652-656  of  Polak  (1997).  We  begin  with  the  definitions 
related  to  continuity. 

Definition  VII.  1.  Let  V  be  a  real  norrned  space  and  let  S  be  a  convex  subset  of  V. 

(i)  A  function  /  :  V  — >  Mm  is  said  to  be  continuous  at  a  point  x  G  V,  if,  for  every 
S  >  0,  there  exists  a  p  >  0  such  that 

\\f(x')-f(x)\\  <6,W  e°B(x,P),  (VII. 1) 

o 

where  B(x,  p)  =  {x'  G  Mn  j  \\x'  —  x||  <  p}.  A  function  /  :  V  — >  is  said  to  be 

continuous  (continuous  on  S )  if  it  is  continuous  at  all  x  G  V  (x  E  S). 

(ii)  A  function  /  :  V  — >  Mm  is  said  to  be  continuous,  relative  to  S  (S'-continuous), 
if,  for  every  x  G  S  and  for  every  <5  >  0,  there  exists  a  p  >  0  such  that 

11/00  -  /(Oil  <  G  B{x,p)  n  S.  (VII. 2) 

□ 

Next  we  state  definitions  related  to  differentiability. 

Definition  VII. 2.  Let  V  be  a  real  nornred  space. 

(i)  We  will  say  that  a  continuous  function  /  :  V  — >  Mm  is  Gateaux  differentiable  at 
a  point  x*  G  V,  if  there  exists  a  bounded  linear  operator  fx(x*)  :  V  — >  Mm  such 
that,  for  every  5x  G  V, 

fim  ll/P  +  Afa)  ~  f(x*)  -  Xfx{x*)5x\\  =  Q 

a;o  A  ■  V  ■  ) 

We  will  call  fx(x*)  the  Gateaux  derivative  of  /(•)  at  x*. 

We  will  say  that  a  continuous  function  /  :  V  — >  Mm  is  Gateaux  differentiable  on 
a  subset  S  of  V,  if  it  is  Gateaux  differentiable  at  all  x  G  S. 
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(ii)  We  will  say  that  a  continuous  function  /  :  V  — >  is  Frechet  differentiable  at 

a  point  x*  G  V,  if  it  is  Gateaux  differentiable  at  x*,  with  Gateaux  derivative 
fx(x*),  and,  in  addition,  the  Gateaux  derivative  has  the  property  that 


lim 

Sx— >-0 


1|  f(x*  +  Sx)  -  f(x*) 

IIM 


fx(x*){Sx)  II 


0. 


(VII. 4) 


In  this  case,  we  will  also  call  fx(x*)  the  Frechet  derivative  of  /(•)  at  x*. 

We  will  say  that  a  continuous  function  /  :  V  — >■  Mm  is  Frechet  differentiable  on 
a  subset  S  of  V,  if  it  is  Frechet  differentiable  at  all  x  €  S. 


(iii)  Let  S'  C  S  be  two  convex  subsets  of  V.  We  will  say  that  a  continuous  function 
/  :  S  — >  Mm  is  Gateaux  differentiable,  relative  to  S,  (Gateaux  S-differentiable) 
at  a  point  x*  G  S',  if  there  exists  a  bounded  linear  operator  fx(x*)  :  V  — >•  Mm 
such  that,  for  every  Sx  G  V  such  that  x*  +  A *Sx  G  S,  for  some  A*  >  0, 


gm  II  f(x*  +  A  Sx)  -  f(x*)  -  A  fx(x*)5x 
A4-0  A 


(VII. 5) 


We  will  call  fx(x*)  the  Gateaux  , S'- derivative  of  /(•)  at  x*. 

We  will  say  that  /(•)  is  Gateaux  S'-differentiable  on  S',  if  it  is  Gateaux  S- 
differentiable  at  all  x  G  S' . 


(iv)  Let  S'  C  5  be  two  convex  subsets  of  V.  We  will  say  that  a  continuous  function 
/  :  S  — >  Mm  is  Frechet  differentiable,  relative  to  S,  (S'-differentiable)  at  a  point 
f  G  S',  if  it  is  Gateaux  S’-differentiable  at  x*,  with  Gateaux  S-derivative  fx(x*), 
and,  in  addition,  the  Gateaux  S-derivative  has  the  property  that 


lim 

Sx— >-0 
x*+5x£S 


II  f(x*  +  Sx)  -  f(x*) 

ll^ll 


fx(x*)(Sx)  || 


0. 


(VII. 6) 


In  this  case,  we  will  also  call  fx{x*)  the  Frechet  S-derivative  of  /(•)  at  x*. 


We  will  say  that  /(•)  is  S-differentiable  on  S',  if  it  is  Frechet  S-differentiable  at 
all  a;  G  S'. 


□ 

The  next  two  paragraphs  are  a  restatement  of  the  beginning  of  Section  5.6.2 
from  Polak  (1997),  but  are  included  here  for  the  sake  of  completeness  to  provide  a 
detailed  explanation  of  what  it  means  for  a  function  to  be  continuous  relative  to  the 
set  H  and  Gateaux  differentiable  at  rj  G  H11,  relative  to  the  set  H. 
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Based  on  Definition  VII. 1,  we  can  now  give  a  detailed  explanation  of  what  it 
means  for  a  function  to  be  continuous  relative  to  the  set  H.  Let  S  be  a  convex  subset 
of  H.  Then,  according  to  Definition  VII.  1,  a  function  /  :  S  — y  Mn  is  continuous 
relative  to  H.  at  a  point  rj  G  S,  if,  for  any  sequence  {t^}^ 0,  with  r/,  G  H  for  all 
i  £  N,  such  that  rjl  — »  rj,  as  i  — >  oo,  f{rji)  f{rj),  as  i  — >  oo.  It  should  be  noted, 
however,  that  if  {rji}^Z0  is  any  arbitrary  sequence  converging  to  rj,  then  it  is  possible 
that  r)i  H  for  all  i  G  N.  Then,  we  cannot  claim  that  f(rji)  — >  f(rj),  as  i  — >  oo:-, 
and  hence  we  cannot  necessarily  conclude  that  /(•)  is  continuous  at  rj.  To  keep  our 
terminology  as  concise  as  possible,  as  in  Definition  VII.  1,  given  S,  a  convex  subset  of 
H.  we  will  say  that  a  function  /  :  S  — »  Mn  is  H-continuous  when  we  mean  that  it  is 
continuous  relative  to  H. 

Similarly,  based  on  Definition  VII. 2  we  can  give  a  detailed  explanation  of  what 
it  means  for  a  function  to  be  Gateaux  differentiable  at  r)  G  H°,  relative  to  the  set  H. 
We  begin  by  noting  that  given  any  rj  G  H°  and  any  5r)  G  H ^2,  there  always  exists 
a  A  >  0  such  that  rj  +  A  Sr)  G  H.  Then  it  follows  from  Definition  VII.2(iii)  that  a 
function  /  :  H  — y  Mn  is  Gateaux  differentiable  at  rj  G  H°,  relative  to  H.  if  there  exists 
a  continuous  linear  map  Df(r) ;  •)  from  H0 0j2  into  Mn,  such  that  for  all  Sr j  G  hfoo,2, 


Um  f(v  +  My)  ~  f(v)  ~  AP/fa  Srj) 
A4-0  A 


(VII. 7) 


As  a  result,  if  /  :  H  — >  Mn  is  Gateaux  differentiable  at  rj  G  H°,  relative  to  H.  it  is  also 
Gateaux  differentiable  at  rj.  It  also  follows  from  Definition  VII. 2 (iv)  that  /  :  H  — > 
is  Frechet  differentiable  relative  to  H  (Frechet  H-differentiable)  at  rj  G  H°,  if  it  is 
Gateaux  differentiable  at  rj  and  the  Gateaux  differential  Df(r) ;  •)  has  the  property 


r  II  fW)  -  f(v )  -  Df  (rj;  rj'  -  r;)|| 

,  .!|h  - VWG - =  0' 

n  -v\ \h2^o 


(VII. 8) 


Then  by  definition,  a  function  /  :  H  — )■  Mn  is  Gateaux/Frechet  H-differentiable  on 
H°,  if  it  is  Gateaux/Frechet  H-differentiable  at  every  rj  G  H°. 
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